Reconciliation of data stored on permissioned database storage across independent computing nodes

ABSTRACT

Reconciliation and subscription-model permissions of data stored across independent ledger instances of a database. A system includes a resource manager coupled to a plurality of client accounts. The system includes an execution platform and a shared permissioned ledger comprising independent processing and storage nodes for executing data operations for the plurality of client accounts. The resource manager defines a settlement group comprising one or more client accounts and authenticates an observer node associated with the settlement group. The resource manager assigns ingested data an encryption level on a key hierarchy based on content of the ingested data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/217,956, filed Mar. 30, 2021, which is a continuation-in-part of U.S.patent application Ser. No. 15/701,313, filed Sep. 11, 2017, whichclaims the benefit of U.S. Provisional Patent Application No.62/393,395, filed on Sep. 12, 2016, and claims the benefit of U.S.Provisional Patent Application No. 62/393,399, filed Sep. 12, 2016; andU.S. patent application Ser. No. 17/217,956 is also acontinuation-in-part of U.S. patent application Ser. No. 15/702,684,filed Sep. 12, 2017, which claims the benefit of U.S. Provisional PatentApplication No. 62/393,399, filed on Sep. 12, 2016, and claims thebenefit of U.S. Provisional Patent Application No. 62/393,395, filedSep. 12, 2016; U.S. patent application Ser. No. 17/217,956 is also acontinuation-in-part of U.S. patent application Ser. No. 15/923,961,filed Mar. 16, 2018, which claims the benefit of U.S. Provisional PatentApplication No. 62/472,891, filed Mar. 17, 2017, and which is acontinuation-in-part of U.S. patent application Ser. No. 15/701,313,filed Sep. 11, 2017, which claims the benefit of U.S. Provisional PatentApplication No. 62/393,399, filed on Sep. 12, 2016, and claims thebenefit of U.S. Provisional Patent Application No. 62/393,395, filedSep. 12, 2016; and U.S. patent application Ser. No. 17/217,956 is also acontinuation-in-part of U.S. patent application Ser. No. 15/969,715,filed May 2, 2018, which claims the benefit of U.S. Provisional PatentApplication No. 62/500,314, filed May 2, 2017. The aforementioned patentapplications are incorporated herein by reference in their entireties,including but not limited to those portions that specifically appearhereinafter, the incorporation by reference being made with thefollowing exception: In the event that any portion of theabove-referenced patent applications is inconsistent with thisapplication, this application supersedes the above-referenced patentapplications.

TECHNICAL FIELD

The present disclosure relates to permissioned database storage acrossledger instances and specifically to reconciliation of data entries.

BACKGROUND

Numerous industries generate and output data in high volumes that canbecome unmanageable for ingestion, analysis, and storage. Exampleindustries generating high-volume data entries include, for example,retail industries generating transactional data, financial industriesexecuting trades between parties, research and development industriesreceiving sensor data, and so forth. Traditional systems for dataingestion, analysis, and storage are complicated by processing andstorage constraints. These traditional systems experience high latencywhen different data output nodes provide data at different volume ratesover time. In some industries, it can be critical to ingest and assessenormous sums of data in near real-time.

In some cases, and particularly in the financial transaction industry,it can be important for multiple parties to ingest and assess incomingdata in near real-time. In traditional transactional systems, executinga transaction between two or more counterparties can take several daysbecause each of the two or more counterparties must analyze large sumsof data to determine obligations and exposures before settling thetransaction. The delay in executing transactions between counterpartiesis associated with the latency in processing and storage resources forassessing incoming, up-to-date transactional data. Additionally, mosttransactions can only be executed during business hours when variousentities such as banks, clearinghouses, and exchanges are open andoperating. This is due at least in part to the individualized dataanalysis performed by the counterparties.

What is needed are centralized data ingestion, analysis, and storageresources for multiple parties that cannot share data for securityreasons but must coordinate actions based on data analysis. In light ofthe foregoing, disclosed herein are systems, methods, and devices fornear real-time analysis of information consumed by high-throughput dataingestion. Additionally, disclosed herein are systems, methods, anddevices for partitioning data in a database for secured, permissionedaccess such that the data can be assessed by independent nodes forcoordinating actions between parties.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the present disclosureare described with reference to the following figures, wherein likereference numerals refer to like parts throughout the various figuresunless otherwise specified.

FIG. 1 is a schematic block diagram of a system for data management,permissioned access to database entries, and load-balancing ofprocessing resources;

FIG. 2 is a schematic block diagram illustrating example components,modules, and functionality of a resource manager as described herein;

FIG. 3 is a schematic diagram illustrating communication between aresource manager and a shared permissioned ledger comprising a pluralityof client ledger instances;

FIG. 4 is a schematic diagram of a system for data ingestion, userauthentication, and granting access to data stored across a plurality ofclient ledger instances on a shared permissioned ledger;

FIG. 5 is a schematic diagram of a system for data ingestion comprisingnode-specific ingestors and node-specific normalizers for ingesting andprocessing data received from independent data stream event channels;

FIG. 6 is a schematic diagram of a system for data ingestion andreal-time data processing;

FIG. 7 is a schematic diagram of a system and process flow for dataingestion and calculating obligations and exposures of parties in nearreal-time based on the ingested data;

FIG. 8 is a schematic diagram of a system for trade settlement amongstmembers of a settlement group;

FIG. 9 is a schematic diagram of a system for settling trades betweenmembers of a settlement group based at least in part on outputs from asettlement module;

FIG. 10 is a schematic diagram of a system and process flow forexecuting an asset transfer between client accounts and/or an externalaccount;

FIG. 11 is a schematic diagram of a system for executing an assettransfer between two financial institutions using different currencies;

FIG. 12 is a schematic diagram of a system for pushing data by way of asubscription model to authorized nodes of a shared permissioned ledger;

FIG. 13 is a schematic diagram of a system for pushing data events toparticipants and observers according to a subscription model;

FIG. 14 is a schematic diagram of a hash tree that can be implementedwith the methods and systems described herein for securing andorganizing data on the shared permissioned ledger;

FIG. 15 is a schematic diagram of a multiple level key hierarchy;

FIG. 16 is a schematic diagram of an example hierarchy for groupingtrades;

FIG. 17 is a schematic diagram of a system for asset transfer betweensettlement banks;

FIG. 18 is a schematic diagram of a system for communications between ashared permissioned ledger and outside parties;

FIG. 19 is a schematic diagram of a system for data security andpermissioned database access;

FIG. 20A is a schematic diagram of a prior art system for executingtrades between parties;

FIG. 20B is a schematic diagram of a system for improved transactionexecution and streamlined communications between parties;

FIG. 21 is a schematic diagram of a system for communications betweencomputing nodes and a distributed transaction store;

FIG. 22 illustrates an example of bilateral netting calculations;

FIG. 23 illustrates an example of multilateral netting calculations;

FIG. 24 is a schematic diagram of a settlement group comprising aplurality of client accounts, wherein trades within the settlement groupare overseen and managed by a resource manager;

FIG. 25 is a schematic diagram illustrating example data entries storedon independent client ledger instances of a shared permissioned ledgerover the course of execution of a trade;

FIG. 26 is a schematic diagram of a system and process flow for matchingtrades and determining whether a trade has been fully settled;

FIG. 27 is a schematic diagram of a system and process flow fornegotiating a trade split between counterparties;

FIG. 28 is a schematic diagram of a system for centralized facilitationof financial transactions between parties;

FIG. 29 illustrates an example state diagram showing various states thata trade may undergo from initiation to settlement;

FIG. 30 is a schematic flow chart diagram of a method for transferringassets between funds in different currencies;

FIG. 31 is a schematic flow chart diagram of a method for initiating aworkflow and generating reconciliation data applicable to the workflow;

FIG. 32 is a schematic flow chart diagram of a method for transferringassets between financial institutions;

FIG. 33 is a schematic flow chart diagram of a method for authenticatinga client and validating a transaction;

FIG. 34 is a schematic flow chart diagram of a method for nettingtransactions; and

FIG. 35 is a schematic block diagram illustrating example components ofa computing device.

DETAILED DESCRIPTION

Disclosed herein are systems, methods, and devices for reconciliationand subscription-model permissions of data stored on permissioneddatabase storage across independent computing nodes. A system disclosedherein includes a resource manager in communication with a network, anexecution platform comprising a plurality of processing nodes, and ashared permissioned ledger comprising a plurality of ledger instances.The resource manager scales the processing resources and storageresources independently of one another across a plurality of clientaccounts based on need. The resource manager defines a settlement groupcomprising one or more settlements clients and ingests data for each ofthe one or more settlement clients. The resource manager assignspermissions to ingested data and provides data to authenticatedobservers by way of a push or pull data model. The resource managerreconciles data stored on the shared permissioned ledger.

The systems, methods, and devices disclosed herein leverage scalableprocessing and storage resources for near real-time analysis of enormoussums of data consumed by high-throughput data ingestion. The consumeddata is analyzed in near real-time to calculate metrics for a pluralityof unrelated entities such that the unrelated entities can make informeddecisions based on up-to-date data analysis. Additionally, the consumeddata is normalized, partitioned, and replicated across multiple ledgerinstances of a shared permissioned ledger such that the unrelatedparties can query their own ledger instances to read their own datawhile ensuring their data is not accessible to other parties withoutexpress authorization.

The systems, methods, and devices disclosed herein reduce latency indata analysis and database management systems by leveraging independentscalability of processing and storage resources. Additionally, thesystems, methods, and devices disclosed herein enable efficient,low-latency ingestion of enormous sums of data from multiplesource-nodes, wherein the data can be ingested in different proprietaryformats and normalized into a standard canonical format being storedacross an independent database ledger instances. The data ingestion,normalization, and analysis can be executed by dedicated, node-specificingestion, normalization, and analysis nodes for each incoming datachannel. The processing and storage resources for the node-specificingestion, normalization, and analysis nodes can be scaled up and downacross the system based on need.

The systems, methods, and devices disclosed herein can be implemented ina centralized trade management system, although it should be appreciatedthat the disclosures presented herein represent computer-basedimprovements applicable to numerous industries. Specifically, thesystems and methods described herein can be implemented in a distributedtrading platform for tracking, managing, and executing transactionsbetween parties based on real-time liquidity metrics. Thecomputer-centric improvements described herein allow for nearlyinstantaneous transaction settlement between parties.

Referring now to the figures, FIG. 1 is a schematic diagram of a system100 for data management and permissioned access to database entries. Thesystem includes a resource manager 102 in communication with a network118. The resource manager 102 is executed by one or more servers incommunication with the components illustrated in FIG. 1 , such as thesystem data 116, shared metadata 114, client accounts 104, executionplatform 106 processing nodes, and shared permissioned ledger 110 clientledger instances 112. In an implementation, the resource manager 102 isadditionally in communication with financial institutions, exchanges,clearinghouses, and so forth for executing complex financialtransactions for multiple parties.

The resource manager 102 oversees data ingestion and data management fora plurality of client accounts 104, such as client account A 104 a,client account B 104 b, and client account C 104 c (may generically bereferred to as client account 104).

The resource manager 102 manages an execution platform 106 that includesa plurality of processing nodes 108 associated with the client accounts104. The client accounts 104 may share the processing resources of theexecution platform 106 and/or may be assigned independent processingresources. FIG. 1 illustrates a plurality of processing nodes within theexecution platform 106, including processing node A 108 a, processingnode B 108 b, and processing node C 108 c.

The resource manager 102 manages the ingestion, normalization,organization, and storage of data entries within the shared permissionedledger 110. The shared permissioned ledger 110 includes data entriespertaining to transactions associated with the client accounts 104. Theclient accounts 104 may have secure, permissioned access to data entriesbased on permissions stored in shared metadata 114. The sharedpermissioned ledger 110 includes data entries stored across a pluralityof ledger instances, including, for example, client ledger instance A112 a, client ledger instance B 112 b, and client ledger instance C 112c (may generically be referred to herein as client ledger instance 112).It should be appreciated that the resource manager 102 may be incommunication with any number of client accounts 104, processing nodes108 a-108 b, and client ledger instances 112 a-112 c.

The resource manager 102 detects and records all client metadata on theshared metadata 114 store. This creates an audit trail for the clientmetadata. Trade data is not stored on the shared metadata 114 store andis instead stored on the shared permissioned ledger 110. The sharedmetadata 114 store includes information about, for example, the lifecycles of accounts, thresholds defined by client accounts 104,rules-based triggers defined by client accounts 104 and/or the resourcemanager 102, bank accounts, financial institution identifiers, currencydata, market data, and so forth.

In an embodiment, the shared metadata 114 only stores data that ismarked to be shared publicly between client accounts 104. The sharedmetadata 114 store includes, for example, name and public identifier ofthe client account 104 along with other identifiers the client account104 might utilize on incoming trade data. The shared metadata 114 storemay be implemented as a common store that is stored separately from theshared permissioned leger 110 and the client ledger instances 112. Theshared metadata 114 store serves as a directory service to providelookups for identifying participants.

The shared permissioned ledger 110 stores data in partitions that can bequeried by the resource manager 102. The data entries in the sharedpermissioned ledger 110 are immutable such that the entries cannot bedeleted or modified and can only be replaced by storing a new,superseding data entry. The data stored in the shared permissionedledger 110 is auditable. In particular implementations, replicated datastore 114 is an append only data store which keeps track of allintermediate states of the transactions. Additional metadata may bestored along with the transaction data for referencing informationavailable in external systems. In specific embodiments, replicated datastore 114 may be contained within a financial institution or othersystem.

In an implementation, the shared permissioned ledger 110 stores dataentries comprising transaction information for client accounts andoutside entities. The shared permissioned ledger 110 includes anindependent client ledger instance 110 associated with each clientaccount. The independent client ledger instances 110 may includeseparate and unique hardware for storing ledger data entries. In anembodiment, the shared permissioned ledger 110 includes scalable storagehardware that can be partitioned to client ledger instances 112 based onneed. The resource manager 102 can allocate additional storage spacewithin the shared permissioned ledger 110 based on which client account104 requires additional storage. The storage space within the sharedpermissioned ledger 110 is independently scalable of the processingresources within the execution platform 106. This enables numerousbenefits and ensures that each client account 104 has access tosufficient processing resources and storage resources based on need.

The independent client ledger instances 112 store different versions ofthe shared permissioned ledger 110 based on the permissions for thecorresponding client account. The shared permissioned ledger 110holistically stores transaction state information for all transactionsrequested, pending, and settled by all entities within the resourcemanager's 102 network of client accounts 104. However, differentversions of the shared permissioned ledger 110 are stored on theindependent client ledger instances 112 based on need. In an embodiment,the system 100 does not include a single, centralized copy of all dataentries within the shared permissioned ledger 110. Instead, the dataentries within the shared permissioned ledger 110 are dispersed amongstthe client ledger instances 112 based on client account permissionsstored in shared metadata 114. Each client account 104 has access onlyto the client ledger instance 112 associated with that client account.For example, client account A only has access to data entries stored onclient ledger instance A; client account B only has access to datastored on client ledger instance B; and client account C only has accessto data entries stored on client ledger instance C. The resource manager102 manages access to the data entries stored on the client ledgerinstances and ensures, for example, that processing node A cannot reador write to the client ledger instance C.

The shared permissioned ledger 110 is modeled after double-entryaccounting principles. When a transaction is initiated betweencounterparties, at least two data entries are stored on the sharedpermissioned ledger 110, including a first data entry stored on a firstclient ledger instance associated with the first party to thetransaction, and a second data entry stored on a second client ledgerinstance associated with the second party to the transaction. The firstdata entry and the second data entry are not necessarily duplicates ofone another and may include different information that is applicable tothe corresponding party to the transaction. In some cases, the firstdata entry and the second data entry are duplicates of one another.Nevertheless, the processing node associated with the first party cannotread the second data entry stored on the client ledger instanceassociated with the second party. The first party does not havepermissions to query, read, or write to the client ledger instanceassociated with the second party (and vice versa).

In some cases, the resource manager 102 causes only one data entry (withno duplicate) to be stored on one client ledger instance of the sharedpermissioned ledger 110. This occurs when the content of the data entryis applicable to only one client account. For example, if a first partyto a transaction has completed an internal review of a transaction,cleared funds for the transaction, etc., and this update does notinvolve the second party to the transaction, then the resource manager102 may cause a data entry to be stored on the client ledger instanceassociated with the first party that includes the applicableinformation. In this case, the resource manager 102 does not cause aduplicate data entry to be stored on the client ledger instanceassociated with the second party to the transaction because the secondparty does not have permission to view the applicable information and/orthe applicable information is not relevant to the second party. As partof the normalization process, the resource manager 102 identifies theprincipals of a trade. The trades are replicated and stored on allparticipant client ledger instances 112. Generally, incoming data isstored to the client ledger instances 112 of all participants to thetrade.

The shared permissioned ledger 110 can be implemented to store anauditable trail of financial transaction data. A data entry comprisingfinancial transaction data includes one or more of data related to theprincipal parties to the transaction, a transaction date, a transactionamount, a transaction state (e.g., initiated, pending, cleared,settled), relevant workflow references, a trade ID, a transaction ID,and additional metadata to associate the transaction(s) with one orexternal systems. The data entries stored on the shared permissionedledger 110 include cryptographic hashes to provide tamper resistance andauditability.

In some embodiments, the shared permissioned ledger 110 is a sharedledger that can be accessed by multiple financial institutions and othersystems and devices. In particular implementations, both parties to aspecific transaction can access all details related to that transactionstored in the shared permissioned ledger 110. All details related to thetransaction include, for example, the parties involved in thetransaction, the type of transaction, the date and time of thetransaction, the amount of the transaction, and other data associatedwith the transaction. In some embodiments, each transaction entry storedon the shared permissioned ledger 110 includes a client identifier, ahash of the transaction, an initiator of the transaction, and a time ofthe transaction.

The resource manager 102 allows for selective replication of data storedon the shared permissioned ledger. The resource manager 102 can permitan outside entity (e.g., a bank, financial institution, clearinghouse,exchange, and so forth) to replicate certain data entries stored on theshared permissioned ledger 110. The resource manager 102 does not allowany outside party to query, read, or write to any data stored on aclient ledger instance without receiving express authorization. In anexample implementation, client account A wishes to release certain dataentries to an outside party. Client account A communicates with theresource manager 102 and indicates that the resource manager 102 shouldpermit the outside party to query, read, and/or write to the certaindata entries. The resource manager 102 then grants the outside partyaccess to the certain data entries stored on the client ledger instanceA.

The processing nodes 108 for each client account can calculate theoverall obligations, exposures, and liquidity for that client account inreal-time based on data stored on the shared permissioned ledger 110.The resource manager 102 is notified when a trade is initiated andgenerates an auditable trail of data entries for the lifetime of thattrade. The resource manager 102 causes data entries to be stored on theshared permissioned ledger 110 whenever the trade undergoes a statechange. The processing nodes 108 can reference the shared permissionedledger to calculate the obligations, exposures, and liquidity of aclient account in real-time because current transaction information iscontinually stored on the shared permissioned ledger 110.

The network 118 includes any type of network, such as a local areanetwork, a wide area network, the Internet, a cellular communicationnetwork, or any combination of two or more communication networks. Theresource manager 102 communicates with some client accounts 104 andoutside parties by way of communication protocols such as SWIFT MT(Society for Worldwide Interbank Financial Telecommunication MessageType) and proprietary application interfaces. The resource manager 102ingests data and receives communications from client accounts 104 (andentities associated with the client accounts 104) using secure APIs(Application Program Interfaces) and other protocols. The resourcemanager 102 can integrate with existing financial institutions, banks,clearinghouses, and exchanges without significant modification to theinstitution's systems.

The network 118 may include a data communication network, such as alocal area network, wide area network, the Internet, a cellularcommunication network, or any combination of two or more communicationnetworks. The resource manager 102 and other components described hereinmay communicate by way of the network 118 in a suitable communicationprotocol. The resource manager 102 may communicate with externaldatastores associated with the client accounts 104 by way of SWIFT MT(Society for Worldwide Interbank Financial Telecommunication MessageType) messages such as MT 2XX, 6XX, 9XX. The resource manager 102 maycommunicate with ISO 20022 (a standard for electronic data interchangebetween financial institutions), and proprietary application interfacesexposed by particular financial institutions. Each of the clientaccounts 104 may be associated with a financial institution such as abank, exchange, hedge fund, or other type of financial entity or system.

In an implementation, the resource manager 102 oversees and managestrades between client accounts 104 and outside parties. Because theresource manager 102 is in communication with the shared permissionedledger 110, the resource manager 102 can calculate liquidity and overallobligations and exposures for each of the client accounts 104 inreal-time. This enables the resource manger 102 to settle financialtransactions even when exchanges and clearinghouses are closed. Thus,the resource manager 102 can execute a financial transaction nearlyimmediately upon receiving a request to execute the transaction. Thisrepresents a significant improvement over traditional trading systems,wherein a financial transaction may take several days to settle toensure the transaction counterparties have sufficient liquidity.

As discussed in greater detail herein, the resource manager 102 managesasset transfers between numerous entities. In many cases, execution ofan asset transfer includes the use of a central bank to clear and settlethe funds. The central bank provides financial services for a country'sgovernment and commercial banking system. In the United States, thecentral bank is the Federal Reserve Bank. In some implementations,resource manager 102 provides an on-demand gateway integrated into theheterogeneous core ledgers of financial institutions (e.g., banks) toview funds and clear and settle all asset classes. The resource manager102 may also settle funds using existing services such as FedWire.

The resource manager 102 communicates with authorized systems andauthorized users. The authorized set of systems and users often resideoutside the jurisdiction of the resource manager 102. Typically,interactions with these systems and users are performed via securedchannels such as SWFIT messaging and/or secure APIs. To ensure theintegrity of the resource manager 102, various constructs are used toprovide system/platform integrity as well as data integrity.

In an embodiment, the system data 116 database stores a listing ofauthorized machines, devices, and accounts (i.e., “whitelisted”). Theresource manager 102 accesses the system data 116 to determine whether auser is authorized, and what data that user is authorized to access. Theresource manager 102 verifies the identity of each machine usingsecurity certificates and cryptographic keys. The resource manager 102securely communicates with outside parties by way of secure API accesspoints. The resource manager 102 stores a listing of authorized usersand roles, which may include actual users, systems, devices, orapplications that are authorized to interact with resource manager 102and/or access certain data stored on the shared permissioned ledger 110.System/platform integrity is also provided through the use of securechannels to communicate between resource manager 102 and externalsystems. In some embodiments, communication between the resource manager102 and external systems is performed using highly secure TLS (TransportLayer Security) with well-established handshakes between the resourcemanager 102 and the external systems. Particular implementations may usededicated virtual private clouds (VPCs) for communication between theresource manager 102 and any external systems. Dedicated VPCs offerclients the ability to set up their own security and rules for accessingresource manager 102. In some situations, an external system or user mayuse the DirectConnect network service for better service-levelagreements and security.

The resource manager 102 allows each client account 104 to configure andleverage their own authentication systems. This allows clients toestablish custom policies on user identity verification, includingtwo-factor authentication and account verification.

The resource manager 102 supports role-based access control of workflowsand the actions associated with workflows. Example workflows may includePayment vs Payment (PVP) and Delivery vs Payment (DVP) workflows. Insome embodiments, users can customize a workflow to add custom steps tointegrate with external systems that can trigger a change in transactionstate or associate them with manual steps. Additionally, systemdevelopers can develop custom workflows to support new businessprocesses. In particular implementations, some of the actions performedby a workflow can be manual approvals, a SWIFT message request/response,scheduled or time-based actions, and the like. In some embodiments,roles can be assigned to particular users and access control lists canbe applied to roles. An access control list controls access to actionsand operations on entities within a network. This approach provides ahierarchical way of assigning privileges to users. A set of roles alsoincludes roles related to replication of data, which allows the resourcemanager 102 to identify what data can be replicated and who is theauthorized user to be receiving the data at an external system.

Additionally, one or more rules identify anomalies which may trigger amanual intervention by a user or principal to resolve the issue. Exampleanomalies include system request patterns that are not expected, such asa high number of failed login attempts, password resets, invalidcertificates, volume of requests, excessive timeouts, http errors, andthe like. Anomalies may also include data request patterns that are notexpected, such as first time use of an account number, significantlylarger than normal number of payments being requested, attempts to movefunds from an account just added, and the like. When an anomaly istriggered, the resource manager 102 is capable of taking a set ofactions. The set of actions may initially be limited to pausing theaction, notifying the principals of the anomaly, and only resumingactivity upon approval from a principal.

FIG. 2 is a schematic block diagram illustrating example modules andfunctionalities of the resource manager 102. In an implementationwherein the resource manager 102 oversees and manages financialtransactions for certain parties, the resource manager 102 cancommunicate directly with one or more Central Counterpart Clearinghouses(CCPs), exchanges, banks, asset managers, hedge funds, or data ingestionengines. CCPs are organizations that facilitate trading in variousfinancial markets. Exchanges are marketplaces in which securities,commodities, derivatives, and other financial instruments are traded.Banks include any type of bank, credit union, savings and loan, or otherfinancial institution. Asset managers include asset managementorganizations, asset management systems, and the like. In addition tohedge funds, the resource manager 102 may also communicate with othertypes of funds, such as mutual funds. The resource manager 102communicates with CCPS, exchanges, banks, asset managers, and hedgefunds using a suitable and secure communication network andcommunication protocol.

The resource manager 102 includes secure APIs 202 that are used bypartners to securely communicate with the resource manager 102. In someembodiments, the secure APIs 202 are stateless to allow for automaticscaling and load balancing. The resource manager 102 scales based onnumerous factors, including the rate of incoming requests and the timeof day to correspond with settlement and cutoff windows. During higherrates, services scale up to provide larger capacity for the processingnodes 108 to process the requests for their respective client accounts104. The resource manager 102 load balances the request across theprocessing nodes 108 and client ledger instances 112 to ensure noindividual instance is overlooked. When the rate returns to normal, theresource manager 102 scales down to keep optimum usage of resources andcost.

The role-based access controller 204 provide access to modules, data,and activities based on the roles of an individual user or participantinteracting with the resource manager 102. In some embodiments, usersbelong to roles that are given permissions to perform certain actions.The resource manager 102 may receive an API request and check the APIrequest against the role to determine whether the user has permissionsto perform an action.

The onboarding module 206 includes the metadata associated with aparticular financial institution, such as bank account information, userinformation, roles, permissions, settlement groups, assets, andsupported workflows. The onboarding module 206 includes functionalityfor authenticating ownership of a bank account or other account.

The onboarding module 206 of the resource manager 102 authenticatesclients during onboarding and validates transactions. The resourcemanager 102 receives a connection request from a node such as afinancial institution, authorized system, authorized user device, and soforth. The resource manager 102 authenticates the node and, ifauthenticated, acknowledges the node as known. The resource manager 102receives a login request from the node to connect to a client account.In response to the login request, the resource manager 102 generates anauthentication token and communicates the authentication token to thenode. The authentication token may be used to determine the identity ofthe user for future requests, such as fund transfer requests. Theidentity is further checked for permissions to various services oractions.

The resource manager 102 receives a transaction request from the node,such as a request to transfer assets between two financial institutionsor other entities. In response to the received transaction request, theresource manager 102 verifies the node's identity and validates therequested transaction. The node's identity may be validated based on anauthentication token, and then the resource manager 102 checkspermissions to determine whether the node has permissions to perform aparticular action or transaction. The transfer of assets furtherincludes validating approval of an account by multiple roles to avoidcompromising the network. If the node's identity and requestedtransaction are verified, the resource manager 102 creates one or moreledger entries on the client ledger instance 112 to store details of thetransaction. The resource manager 102 sends acknowledgement regardingthe transaction to the node with a server transaction token. The servertransaction token may be used at a future time by the client accountwhen conducting audits. The resource manager 102 initiates thetransaction using the systems and methods described herein.

The clearing module 208 includes functionality to transfer assetsbetween accounts within a financial institution. As used herein, DCCrefers to a direct clearing client or an individual or institution thatowes an obligation. A payee refers to an individual or institution thatis owed an obligation. A CCG (or Guarantor) refers to a client clearingguarantor or an institution that guarantees the payment of anobligation. A CCP refers to a central counterparty clearinghouse and aClient is a customer of the FCM (Futures Clearing Merchant or FuturesCommission Merchant)/CCG guarantor. Collateral settlements refer tonon-cash based assets that are cleared and settled between CCP, FCM/CCGguarantor, and DCC. CSW refers to collateral substitution workflow,which is a workflow used for the pledging and recall (includingsubstitution) of collateral for cash. A settlement group refers to alogical grouping of stakeholders who are members of that settlementgroup that are involved in the clearing and settlement of one or moreasset types. A workflow, when executed, facilitates a sequence ofclearing and settlement instructions between members of a settlementgroup as specified by the workflow parameters.

The settlement module 210 monitors and manages the settlement of fundsor other types of assets associated with one or more transactionshandled by the resource manager 102. Settlement execution includesexecuting a complex workflow for managing data and asset transfersbetween parties.

The resource manager 102 and the system 100 provide a unique improvementto computer-based communications and data storage that can be leveragedparticularly in the financial transaction industry for (a) increasingthe speed with which transactions can be executed; (b) increasing thereliability of liquidity metrics; (c) increasing the reliability of riskmetrics; and (d) enabling the obligations and exposures of parties to becalculated in near real-time based on incoming data streams. Because ofthe structure of the system 100, the settlement module 210 of theresource manager 102 is capable of initiating bidirectional movement ofassets in capital markets. The settlement module 210 can finalize atransaction within minutes, and in some cases as quickly as two minutes.This is a significant improvement over traditional systems, whichrequire 24-48 hours to fully settle a transaction. This significantincrease in settlement type is enabled by the structure of the system100 and the communications the resource manager 102 has with outsideparties, client accounts, client processing nodes, and the sharedpermissioned ledger 110.

In some implementations, the settlement module 210 operates under anumber of rule-based triggers to initiate settlements based on certaincircumstances. For example, the settlement module 210 may include arule-based trigger to initiate all pending, valid settlements one-hourbefore cutoff time for an exchange. The rule-based triggers for thesettlement module 210 can be automated, manually configured, and/orsuggested by a neural network that is trained to predict risk andsuggest settlement triggers based on the predicted risk.

The settlement module 210 enables authorized users to execute complexworkflows to enable institutions to move assets on demand. Thesettlement module 210 may additionally allow one or more third partiesto view and confirm payment activities between parties. The settlementmodule 210 enables on-demand settlements across multiple parties basedon near real-time liquidity analysis, even when markets are closed.

A workflow describes the sequence of activities associated with aparticular transaction, such as an asset transfer. The settlement module210 provides a clearing and settlement gateway between multipleentities, e.g., different banks, mutual funds, hedge funds, and soforth. When a workflow is executed, the settlement module 210 generatesand issues clearing and settlement messages (or instructions) tofacilitate the movement of assets. The shared permissioned ledger 110tracks asset movement and provides visibility to the parties andobservers in substantially real time. The integrity of these systems andmethods is important because the systems are dealing with core paymentsthat are a critical part of banking operations. Additionally, many assetmovements are final and irreversible. Therefore, the authenticity of therequest and the accuracy of the instructions are crucial. Further,reconciliation of transactions between multiple parties are important tothe management of financial data.

Payments between parties can be performed using multiple asset types,including, for example, currencies, treasuries, securities such asnotes, bonds, bills, and equities, and the like. Payments can be madefor different reasons, such as margin movements, collateral pledging,swaps, delivery, fees, liquidation proceeds, and the like. As discussedherein, each payment may be associated with one or more metadata.

The settlement module 210 may additionally trigger reconciliation andregulatory reporting for executed trades. In capital markets, assetmovement is triggered due to a settlement on a set of trades betweenparties. All parties involved in the trade as well as the clearing andsettlement of the trade need to perform post trade activities thatinclude reconciliation and regulatory reporting of the trades as well asthe payments associated with the trades. In traditional systems,reconciliation and regulatory reporting is a significant pain point foroperations teams because it is mostly manual and labor intensive. Themain problems related to reconciliation and the regulatory reporting arethe heterogeneous systems that are involved in traditional transactiondata systems.

In many implementations, the number of trade events that occur in a dayis three to five orders of magnitude greater than the number ofsettlements that occur in a day. The settlement module 210 captures thetrade events and determines if a trade has been completed or fullysettled. This simplifies the reconciliation and regulatory reportingproblems experienced by institutions, users, and the like.

The ledger manager 212 manages the shared permissioned ledger 110.Traditional financial institutions typically maintain accountinformation and asset transfer details in a ledger at the financialinstitution. The ledgers at different financial institutions do notcommunicate with one another and often use different data storageformats or protocols. Thus, each financial institution can only accessits own ledger and cannot see data in another financial institution'sledger, even if the two financial institutions implemented a commonasset transfer. The shared permissioned ledger 110 described hereinenables secure coordination between principals based on near real-timeliquidity metrics without sacrificing data security.

The shared permissioned ledger 110 includes distributed ledgertechnology (DLT) in a database format that is spread across multiplesystems or sites, such as different institutions and/or differentgeographic areas. In contrast with traditional distributed ledgertechnology (e.g., Blockchains), data stored on the shared permissionedledger 110 described herein cannot be accessed by all parties and is notreplicated by each party. The shared permissioned ledger 110 describedherein is replicated only when necessary for more than one party toaccess the information, e.g., when both counterparties to a transactionrequire a copy of the same information. Other entities who are not aparty to the transaction do not have access to that information. Theledger manager 212 oversees the organization, replication, and access todata entries stored across the plurality of client ledger instances ofthe shared permissioned ledger.

The interchange module 214 communicates with outside parties andfacilitates transaction settlement. The interchange module 214 maycommunicate by way of FedWire, NSS (National Settlement Service), ACH(Automated Clearinghouse), or other suitable means of communication andtransaction settlement. In most jurisdictions, all bank accounts areassociated with an account on a federal or national level. In the UnitedStates, banks each have an account with the Federal Reserve. When two ormore accounts at the same bank seek to trade assets, the trade is easyto execute. However, when two or more accounts at different banks seekto trade assets, the trade settlement is more complex, because assetsmust be debited from an account at the first bank and transferred to anaccount at the second bank. The Federal Reserve serves as the “bank ofbanks,” wherein all banks have an account at the Federal Reserve. Theprocess of moving assets by debiting money from the first bank includesmoving the money to the first bank's account at the Federal Reserve, andthen crediting the second bank's account at the Federal Reserve, and thecrediting the account at the second bank. The interchange module 214communicates with outside parties for executing transactions betweenbanks.

The blockchain module 216 provides interoperability with blockchains forsettlement of assets on a blockchain.

When some financial transactions undergo a state change (e.g.,initiated-pending-approved-cleared-settled, etc.) it may trigger one ormore notifications to the parties involved in the transaction. Thesystems and methods described herein provide multiple ways to receiveand respond to these notifications. In some embodiments, thesenotifications can be viewed and acknowledged using a dashboardassociated with the described systems and methods or using one or moreAPIs.

The database ledger and replication module 218 exposes constructs of theshared permissioned ledger 110 to the resource manager 102. The databaseledger and replication module 218 stores immutable transaction states onthe shared permissioned ledger 110 such that the transaction states, andthe history of a transaction, can be audited by querying the sharedpermissioned ledger 110. The database ledger and replication module 218oversees the replication and authorized read/write of data entriesstored on the shared permissioned ledger 110.

The access manager 220 monitors permissions to data stored on the sharedpermissioned ledger 110 and elsewhere throughout the system 100. In somecases, an outsider who is not a party to a transaction may need accessto information about the transaction. The outsider may be granted“observer” status to information about the transaction. The observer maybe a stakeholder in a transaction or may be involved in the execution ofclearing or settling the transaction. The access manager 220 permits anauthorized observer to subscribe to a subset of notifications associatedwith a transaction. The access manager 220 may grant access uponreceiving authorization from one or more parties to the transaction whoagree the observer can receive the subset of notifications.

The configuration and metadata manager 222 oversees and directs thestorage of metadata and trade data across the shared metadata 114database and the shared permissioned ledger 110.

The resource manager 102 includes or communicates with a data ingestionengine 224. The data ingestion engine 224 includes at least one dataingestion platform that consumes transaction data in real-time alongwith associated events and related metadata. The data ingestion engine224 is a high throughput pipe that provides an ability to ingesttransaction data in multiple formats. The resource manager 102normalizes the ingested data to a canonical format. The normalized datais used by downstream engines like the matching module 226, liquiditymodule 228, optimizers, netting modules, real-time count modules, and soforth.

The matching module 226 is a real-time streaming processor. The matchingmodule 226 identifies multiple data entries and/or transactions thatshould be stitched together as multiple components of a single trade (oranother event). In an embodiment, the matching module 226 is a windowedstream processing component. The matching module 226 can read from anormalized data stream (e.g., trade data in FIXML format) and computethe status of the trade orders. FIG. 15 illustrates an example of aschema produced as a stream by the matching module 226.

The matching module 226 can identify data entries associated withmultiple transactions of a single trade. In some cases, a single tradeis split into multiple smaller “trade-lets,” and each trade-let may beexecuted as a single transaction. It can be important to identify eachof the trade-lets, determine whether the trade-lets have been fullyexecuted, and then determine whether the trade is settled based onwhether each of the trade-lets has been executed.

An example of parsing a trade into multiple transactions (i.e.,trade-lets) is described as follows. In the example, a client initiatesa request to purchase 10,000 shares of IBM stock with a sell sidedealer. The dealer proceeds to execute the order. Often, the order isexecuted in smaller sizes or lots. The smaller transactions are receivedby the back office settlement systems. In this example, assume the orderof 10,000 shares is executed in lot sizes of 2,500 shares each. When itis time to settle the trade, the settlement will occur for all of the10,000 shares (or four executions) at the same time. To ensure the tradesettles completely, the system 100 stitches together all of the uniqueexecutions. Once stitched together, the system 100 can deem that thetrade is ready for settlement.

In some embodiments, the data received by the data ingestion engine 224is for the executions and not for the complete trade order. Theexecutions will each include a unique trade ID. The multiple executionsare identified based on the unique trade ID. In some cases, theretrieval of the multiple executions is a technically complicatedprocess. For example, if the first execution occurred at 10:00 AM andthe second execution occurred at 11:00 AM, the chances that the firstexecution is in memory when the second execution is received are low.The retrieval component of the matching module 226 must determine thatthe order is not complete when the first execution is received and thesystem 100 should prepare to receive another execution of the same tradeorder. The retrieval component additionally determines that the firstexecution needs to be retrieved from the shared database ledger 110 whenthe second execution is received. When the second execution is received,the matching module 226 retrieves the first execution from the shareddatabase ledger and stitches the first execution to the secondexecution. The retrieval component must additionally determine whetherthe system 100 should expect a third execution.

In some cases, subsequent executions (such as the third and fourthexecutions in the above example) do not occur. This is deemed apartially executed trade. The resource manager 102 does not leave thetrade hanging at the close of the trading day. Instead, the resourcemanager 102 completes the trade and marks the trade as having been onlypartially completed.

The following XML-like snipped can be used by the matching module 226.This example is for purposes of illustration and is not representativeof a real trade. The real trade messages are typically proprietary tothe client.

Order <Tx>  <OrderID>23847387DHHHD22</OrderID> <ExecID>667667GttyNm1</ExecID>  <Symbol>AAPL</Symbol> <OrderQty>10000</OrderQty>  <OrderPrice>50</OrderPrice> <Filled>0</Filled>  <ToBeFilled>10000</ToBeFilled> <FillPrice>0</FillPrice>  <TransactionTime>20170323- 10:00:00:000</TransactionTime> </Tx> Execution 1 <Tx> <OrderID>23847387DHHHD22</OrderID>  <ExecID>667667GttyNm2</ExecID> <Symbol>AAPL</Symbol>  <OrderQty>10000</OrderQty> <OrderPrice>50</OrderPrice>  <Filled>2500</Filled> <ToBeFilled>7500</ToBeFilled>  <FillPrice>51</FillPrice> <TransactionTime>20170323-  12:00:00:000</TransactionTime> </Tx>Execution 2 <Tx>  <OrderID>23847387DHHHD22</OrderID> <ExecID>667667GttyNm3</ExecID>  <Symbol>AAPL</Symbol> <OrderQty>10000</OrderQty>  <OrderPrice>50</OrderPrice> <Filled>2500</Filled>  <ToBeFilled>5000</ToBeFilled> <FillPrice>49</FillPrice>  <TransactionTime>20170323- 14:00:00:000</TransactionTime> </Tx> Execution 3 <Tx> <OrderID>23847387DHHHD22</OrderID>  <ExecID>667667GttyNm4</ExecID> <Symbol>AAPL</Symbol>  <OrderQty>10000</OrderQty> <OrderPrice>50</OrderPrice>  <Filled>2500</Filled> <ToBeFilled>2500</ToBeFilled>  <FillPrice>49</FillPrice> <TransactionTime>20170323-  14:00:00:000</TransactionTime> </Tx>Execution 4 <Tx>  <OrderID>23847387DHHHD22</OrderID> <ExecID>667667GttyNm5</ExecID>  <Symbol>AAPL</Symbol> <OrderQty>10000</OrderQty>  <OrderPrice>50</OrderPrice> <Filled>2500</Filled>  <ToBeFilled>2500</ToBeFilled> <FillPrice>49</FillPrice>  <TransactionTime>20170323- 16:00:00:000</TransactionTime> </Tx>

The liquidity module 228 calculates liquidity in near real-time forparties that push trade data to the data ingestion engine 224 of theresource manager 102. The liquidity module 228 calculates overallobligations and exposures and real-time liquidity for all asset typestraded for various parties. In an implementation, the resource manager102 oversees a plurality of settlement groups, wherein each settlementgroup is dedicated to a certain asset-type, such as securities, bonds,certain currencies, and so forth. The liquidity demand for a partyincludes multiple components, including: each counterparty, the assettype being exchanged with each counterparty, and other factors. Thesecomponents are evaluated each time the liquidity demand is calculated orupdated.

In some embodiments, different financial institutions in a distributedenvironment may have different business rules. For example, a particularbusiness rule for a specific financial institution may state that if arisk exposure exceeds a predetermined threshold value (based oncurrency, jurisdiction, etc.), the financial institution needs to takeaction to mitigate its risk, such as generate an alert, force asettlement, open another position with a different counterparty toreduce exposure, and the like.

The liquidity module 228 may additionally calculate risk exposure forvarious parties. The liquidity module 228 executes a statistical modelto predict future obligations and exposures, and calculate predictedrisk based on the future obligations and exposures. The liquidity module228 receives the normalized data from the normalized data channel (firstillustrated as 508 in FIG. 5 ) and executes the statistical model usingdata received from the client in near real-time by way of the normalizeddata channel. The statistical model represents a risk model associatedwith a particular financial institution. The liquidity module 228 mayexecute a different statistical model and/or risk model for each clientaccount based on that client's risk tolerances and risk preferences. Forexample, different financial institutions may use different factors (andapply different weightings to various factors) when determining theirown risk tolerance. These factors include, for example, past historywith a counterparty, current exposures, current obligations, and thelike. Thus, each financial institution's statistical model may generateits own unique risk score.

The liquidity module 228 may apply multiple different statistical modelsto the same data to achieve multiple risk scores. These multiple riskscores are useful to financial institutions for different types oftrades or products. For example, different risk scores may be used forspots versus swaps.

In traditional systems, financial institutions have access to a limitedamount of counterparty data because the data is spread across multipleinternal systems. The systems described herein, and the processesexecuted by the resource manager 102, allow financial institutions toobtain a holistic view of data across all currencies, all jurisdictions,all counterparties, all products, and so forth. The liquidity module 228calculates risk exposure and identifies high-risk counterparties. Thus,a financial institution can identify high-risk counterparties insubstantially real time and act, if necessary, to mitigate riskassociated with the high-risk counterparties.

The system 100 protects proprietary information across the variousclient accounts 104 by executing the liquidity module 228 on independentprocessing nodes 108 assigned to the client accounts 104. The executionplatform 106 may comprise a large sum of processing resources that canbe scaled up and down to the various client accounts 104 based on need.However, the results of the processing executions are not shared acrossthe client accounts 104. This ensures that client data remainsconfidential and that counterparties do not have unwanted insight into aclient's proprietary operations.

In an example implementation, client account A wishes to mitigate riskwhen trading assets with client account B and client account C. Theresource manager 102 oversees data ingestion, analysis, and storage foreach of client account A, client account B, and client account C.Nevertheless, to protect the privacy of all client accounts 104, theresource manager 102 does not permit client account A to viewconfidential data or analyses associated with client accounts B and C.Instead, the resource manager 102 causes the processing node A(associated with client account A) to execute its own instance of theliquidity module 228 to predict risk associated with client accounts Band C. The liquidity module 228 instance associated with client accountA does not have access to confidential information associated withclient accounts B and C (even though that information is stored on theshared permissioned ledger 110). The liquidity module 228 must insteadpredict the risk associated with client accounts B and C based oninformation “owned” by client account A. This information includes, forexample, past interactions with client accounts B and C, predictobligations and exposures associated with client accounts B and C, andknown processes or rules associated with client accounts B and C.

The system and platform integrity is important to the secure operationof the resource manager 102. This integrity is maintained by ensuringthat all actions are initiated by authorized users or systems. When anaction is initiated and the associated data is created, an audit trailof any changes made, and other information related to the action, isrecorded on the shared permissioned ledger 110 for future reference. Inparticular embodiments, the resource manager 102 includes (or interactswith) a roles database and an authentication layer. The roles databasestores various roles of the type discussed herein.

Although particular components are shown in FIG. 2 , alternateembodiments of resource manager 102 may contain additional componentsnot shown in FIG. 2 or may not contain some components shown in FIG. 2 .The resource manager 102 is implemented, at least in part, as acloud-based system. In other examples, resource manager 102 isimplemented, at least on part, in one or more data centers. In someembodiments, some of these modules, components, and systems may bestored in (and/or executed by) multiple different systems. For example,certain modules, components, and systems may be stored in (and/orexecuted by) one or more financial institutions.

FIG. 3 is a schematic diagram of communications between the resourcemanager 102 and the shared permissioned ledger 110. The sharedpermissioned ledger 110 is a database including a plurality of clientledger instances 112, such as client ledger instances 112 a-112 nillustrated in FIG. 3 . Each of the client ledger instances 112 may bestored on independent hardware that is partitioned for a certain clientaccount. In an embodiment, the shared permissioned ledger 110 is storedacross a plurality of storage hardware in a cloud-based database systemin communication with a network. The storage hardware that isholistically assigned to the shared permissioned ledger 110 may then bepartitioned by the resource manager 102 for use by the independentclient ledger instances 112.

In an embodiment, the resource manager 102 scales usage of databasehardware to each of the client ledger instances 112 based on need. Thestorage hardware for the shared permissioned ledger 110 is independentlyscalable from the processing resources of the execution platform 106.The resource manager 102 scales storage and processing resources up anddown to each of the client processing nodes and client ledger instancesbased on need. The resource manager 102 may determine, for example, thatone client account needs additional processing resources but does notrequire additional storage resources at a certain time.

The client ledger instances 112 are partitions of the sharedpermissioned ledger 110. Each of the client ledger instances 112 may bestored on storage hardware located in one geographical location, andthat storage hardware may be in communication with a network 118 to forma cloud-based database platform. Alternatively, the client ledgerinstances 112 may be stored on storage hardware located in a pluralityof geographical locations that collectively make up the sharedpermissioned ledger 110. Each of the client ledger instances 112 storesa different dataset applicable to a certain client account. The clientledger instances 112 do not store duplicate data, and data stored acrossthe shared permissioned ledger is not shared by different client ledgerinstances. If a certain data entry needs to be duplicated for two ormore client accounts (e.g., when two client accounts are counterpartiesto a financial transaction and require the same information about thetransaction), then the data entry will be duplicated and independentlystored on two or more client ledger instances that are associated withthe two or more client accounts.

The resource manager 102 manages permissions for the shared permissionedledger. The resource manager 102 ensures that a client account 104 canonly access the data entries stored on that client account's 104 clientledger instance 112. In some cases, the resource manager 102 may grantspecial permission for a client account 104 or outside party to accessdata entries that do no “belong” to that party. The resource manager 102will only grant special permission after receiving express authorizationto release the data to the other client account or outside party.

Each transaction can have two or more participants. In addition to themultiple parties involved in the transaction, there can be one or more“observers” to the transaction. The observer status is important from acompliance and governance standpoint. For example, the Federal Reserveor the CFTC is not a participant of the transaction but may haveobserver rights on certain type of transactions stored on the sharedpermissioned ledger 110. In some embodiments, the resource manager 102permits outside observers to subscribe to certain types of events.

The shared permissioned ledger 110 replicates a financial institution'sinternal ledger. In some implementations, the shared permissioned ledger110 includes an exact, raw-format duplicate of the financialinstitution's internal ledger, and additionally includes a normalizedversion of the financial institutions internal ledger that includes onlythe required datapoints that can be used by the system 100. Financialinstitutions (i.e., the real-world entities associated with the clientaccounts 104) will never alter their own internal ledgers. The resourcemanager 102 oversees the shared permissioned ledger 110 which serves asa replication of the financial institutions' internal ledger.

The shared permissioned ledger 110 holistically includes informationabout numerous client accounts 104 (i.e., financial institutions such asbanks, hedge funds, clearinghouses, exchanges, and so forth). The sharedpermissioned ledger 110 may include a copy of the data stored in theinternal ledgers of two different financial institutions that may serveas counterparties to a trade. The shared permissioned ledger 110 ispartitioned to store data on independent hardware for each client (i.e.,the client ledger instances 112). The client ledger instances may bevirtually partitioned while stored on the same hardware devices in asingle geographic location. The client leger instances 112 may be spreadacross numerous geographic server locations that are each connected to acloud-based database system.

Each data entry stored on the shared permissioned ledger 110 isassociated with a principal (i.e., a financial institution or a party toa trade). The data entries may additionally include metadata thatindicates who has permission to access the data. The data entries storedon the shared permissioned ledger 110 are immutable such that theycannot be deleted or modified. When a data entry needs to be deleted ormodified, a new data entry is generated that references the obsoletedata entry and includes an indication that the obsolete data entryshould be deleted, and/or that information the obsolete data entryshould be superseded. The resource manager 102 performs analysis on thetrade data stored in the shared permissioned ledger 110 based on themost-recent data entries presumed to include the most up-to-date andaccurate information.

In some cases, a trade undergoes a state change, and only one data entryis stored on the shared permissioned ledger 110 that reflects the statechange. For example, if a trade is in the process of being settled andassets have been transferred out of an account associated with a firstclient and into a suspense account associated with the first client,then the resource manager 102 may cause a data entry to be stored onlyon the client ledger instance 112 associated with the first client, andnot on any other client ledger instance associated with a counterpartyto the trade. The resource manager 102 is generally trained to generatea data entry for an event only for those parties who require informationabout the event. In the example illustrated above, the counterparty tothe trade does not need or have access to information indicating thatthe assets have been transferred from the account to the suspenseaccount associated with the first client. In most cases, when assetsmove out of Bank A, a data entry is stored only on the client ledgerinstance for Bank; and when assets are moved to Bank B, a data entry isstored only the client ledger instance for Bank B.

FIG. 4 is a schematic diagram of a system 400 for data ingestion, userauthentication, and granting access to data stored on the sharedpermissioned ledger 110. The system 400 includes the resource manager102 receiving information from a plurality of data sources, includingclient A data source 402 a, client B data source 402 b, and so forth asneeded. The resource manager 102 ingests the data from the client datasources 402 in a variety of data formats as provided by the clientaccounts. The resource manager 102 normalizes that data and causes thedata to be stored on the shared permissioned ledger 110 in theapplicable client ledger instances 112.

The resource manager 102 causes the data entries to be encrypted priorto storage on the client ledger instances 112. The resource manager 102communicates with a cryptographic service 404 associated with each ofthe client accounts. The resource manager 102 communicates withcryptographic service A 404 a for data associated with client account A,and the resource manager 102 communicates with cryptographic service B404 b for data associated with client account B. The cryptographicservice 404 provides secured access to one or more keys associated witheach client account.

In the example illustrated in FIG. 4 , the cryptographic service A 404 aenables access to three keys associated with client account A. The keysinclude client A level one key 406 a, client A level two key 408 a, andclient A level three key 410 a. Conversely, cryptographic service B 404b enables access to three keys associated with client account B. Thekeys include client B level one key 406 b, client B level two key 408 b,and client B level three key 410 b. The different levels of keys maypertain to different types of data and/or different storage constructswithin the client ledger instance 112.

The cryptographic service 404 may be run directly on the processing node108. In some implementations, the cryptographic service 404 includes asoftware package provided by a third-party that is executed by theprocess node associated with the applicable client account. The resourcemanager 102 routes the node-specific traffic to the applicableprocessing node. The process node uses its own cryptographic key toencrypt the data. Thus, each processing node uses its own cryptographicservice to encrypt the data. Therefore, only the processing nodeassociated with a certain client account can read the data associatedwith that certain client account.

The cryptographic service 404 accesses each client's key stored in theclient key storage and causes the data stored in the shared permissionedledger to be encrypted or de-encrypted as needed. The cryptographicservice 404 ensures security of the data entries stored in the sharedpermissioned ledger 110 using, for example, secure bifurcated keys thatare stored in the client key storage. Each key is unique for theassociated client node. When the resource manager 102 access the sharedpermissioned ledger 110, the resource manager 102 provides a data accessrequest to the cryptographic service 404 that includes the appropriatekey. This ensures that the data access request is authorized.

Cryptographic safeguards are used to detect data tampering in theresource manager 102 and any other systems or devices. Data written tothe shared permissioned ledger 110 and any replicated data may beprotected by one or more of the following, including: stapling allevents associated with a single trade; providing logical connections ofeach commit to those that came before it was made; and immutable dataentries. The logical connections in the shared permissioned ledger 110are also immutable, but principals who are parties to a transaction cansend messages for relinking. In this case, the current and precedinglinks are maintained. For example, trade amendments are quite common. Atrade amendment needs to be connected to the original trade. Forforensic analysis, a bank may wish to identify all trades by aparticular trader. Query characteristics will be graphs, time series,and RDBMS (Relational Database Management System).

The resource manager 102 implements various constructs to ensure dataintegrity. The resource manager 102 implements cryptographic safeguardsto allow a transaction to span 1−n principals. An original trade canpass through multiple lifecycle events where the trade can be split intomultiple sub-trades (may be referred to as sublets). The sub-trades canfurther change ownership due to other contracts and lifecycle events.These changes are captured with cryptographic safeguards to only grantread permissions to portions of the trade (i.e., individual sub-trades)which the party is a principal to. The resource manager 102 createsindependent data entries stored on the shared permissioned ledger 110 torepresent updates for each of the individual sub-trades. The dataentries for the individual sub-trades include cryptographic safeguardsto ensure that only certain parties have read access to thatinformation. The resource manager 102 ensures no other users (other thanprincipals who are parties to a transaction) can view data intransaction. Additionally, no other use has visibility into the data asthe data traverses the various channels. The resource manager 102handles failure scenarios, such as a loss of connectivity in the middleof a transaction. Any data transmitted to a system or device isexplicitly authorized such that each ledger entry can only be seen andread by the principals who were a party to the transaction.

The resource manager 102 implements cryptographic safeguards to detectdata tampering. The data written to the client ledger instances 112 ofthe shared permissioned ledger 110 is protected by one or more of:stapling all events associated with a single transaction, providinglogical connections of each commit to those that came before it wasmade, and ensuring the logical connections are also immutable.Principals can send messages for relinking. The current and precedinglinks are maintained after being updated or relinked by principals tothe transaction. This can be common where a trade amendment is submittedto the original trade. For forensic analysis, a financial institutionmay wish to identify all trades by a particular trader. The querycharacteristics include graphs, time series, and RDBMS.

The resource manager 102 monitors for data tampering. If the sharedpermissioned ledger 110 is compromised in any way and the data isaltered, the resource manager 102 detects precisely what was changed.The resource manager 102 guarantees all participants on the network thatthe data has not been compromised or changed. Information associatedwith changes are made available by way of events such that the eventscan be sent to principals via messaging or presented on a userinterface.

Regarding data forensics, the resource manager 102 is configured todetermine the previous value of an attribute was X, the value is now Y,and the value was changed at time T, by a certain person. If theresource manager 102 is hacked or compromised, there may be any numberof changes to attribute X and all of those changes are captured by theresource manager 102. This ensures that data tampering is discoverableand evident.

FIG. 5 is a schematic diagram of a system 500 for data ingestion. Thesystem includes the data ingestion engine 224 of the resource manager102. The data ingestion engine 224 includes an independent ingestor nodefor each client account. In the example illustrated in FIG. 5 , clientaccount A feeds data to the data ingestion engine 224 by way of client Adata source 402 a, and this data is received by the node A ingestor 504a. Similarly, client account B feeds data to the data ingestion engine224 by way of the client B data source 402 b, and this data is receivedby the node B ingestor; client account B feeds data to the dataingestion engine 224 by way of the client C data source 402, and thisdata is received by the node C ingestor 504 c. The data ingestion engine224 receives data in a variety of formats and languages.

The system consumes data in real-time along with associated events andrelated metadata. The data ingestion engine 224 is a high throughputpipe that ingests data in multiple formats. The data is normalized to acanonical format, which is used by downstream engines. The systemprovides access to information in real-time to different parties of atrade, including calculations such as obligations and exposures of theparticipating parties.

The data ingestion engine 224 feeds the ingested data to the datanormalizer 506 for the data to be normalized to a canonical format thatcan be stored on the shared permissioned ledger 110. The data normalizer506 includes an independent normalizer node for each client account. Inthe example illustrated in FIG. 5 , the data from client account A isingested and fed to the node A normalizer 506 a; data from clientaccount B is ingested and fed to node B normalizer 506 b; and data fromclient account C is ingested and fed to the node C normalizer 506 c. Thedata normalizer 506 partitions independent processing resources for eachnormalizer node. In an embodiment, the data normalizer 506 includes aplurality of processors, and the resource manager 102 scales theprocessing capability up and down based on current need across allclient accounts. In an alternative embodiment, each client account has aset amount of processing resources allocated to data normalization, andthe resource manager 102 does not scale the processing resources inreal-time according to need.

The data ingestion engine 224 is a reliable high-throughput pipe withidempotency such that repeats of the same events do not alter thetransaction data. The data ingestion engine 224 operates withidempotency by identifying unique identifiers associated with eachevent. The data ingestion engine 224 assigns a unique identifier to eachevent processed by the system 500. If the same event is processed again,the data ingestion engine 224 will generate the same unique identifierfor that event. This ensures that the single event is not processedfurther by any other module (e.g., optimizers 410, matching module 226,liquidity module 228, netting module 516, and so forth), and the system500 can operate with idempotency.

The data ingestion engine 224 supports the ability to ingest data indifferent formats from different participants. In some implementations,trade data entries include one or more of the following characteristics.All parties of the trade (principals, broker-dealers, exchanges, etc.)need to get access the information in near real time. A trade has a lifecycle from the point of entry into the system, the execution, theaugmentation of the data in the middle and back offices all the waythrough to the point where the trade is cleared/settled. Sometimes, thetrades may be reversed before it is settled. During this lifecycle,trade metadata is being augmented. The parties of the trade as well asthe banks that act as the custodians of the assets of the principalsfollow a protocol of confirmations and affirmations that are similar toan ACK set in a TCP protocol (with the noted difference that these areasynchronous systems). Trades are of different types and the metadata ofthe trade can change depending on the type of trade. Metadata can bethought of as columns to a row in a csv or fields of attributes in XMLor JSON. The Financial Information eXchange (FIX) protocol (and the xmlversion of it—Fixml) have become standards for the messages to capturethe trade metadata between parties.

The data ingestion engine 224 may receive and categorize incoming dataentries according to the following example protocol. A Node N(i) cantrade with parties M(1) . . . M(N) for various products P(1) . . . P(N).A Trade notation T{(Mi, Ni), Pi} can be used to say that parties Mi andNi have traded a product Pi. In the case of a partial trade, it ispossible that a trade submitted by Ni to Mi may be executed by Mi inseparate batches that aggregate to the whole trade. A trade will resultin several events to be recorded by each party of the trade. Each eventis associated with a set of attributes. By association, these attributesare associated with the trade. Although these attributes are for thetrade T{(Mi, Ni), Pi}, Mi and Ni may not have all the attributes as someattributes may be internal tracking attributes for either Mi or Ni. Thedata ingestion engine 224 ingests these events and the associatedmetadata for an event from both Mi and Ni.

The data normalizer 506 reads the data in the ingestor stream andconverts the data into a standard format. The standard format mayinclude a simplified version of the FIXML standard. The normalized datais pushed to a new stream (the normalized data channel 508) which willbe consumed by downstream modules.

In an example implementation, wherein the data ingestion engine 224 isingesting trade data for financial orders, the incoming messages(orders) ingested by the normalizer are given below:

  babacf6a-0c20-44b9-b837- 5dla28fc0acb,,ABT,1500,100,0,1500,0,05/04/2017 01:55 The corresponding Normalized format is <Order><ClOrdID>babacf6a-0c20-44b9- b837-5dla28fc0acb</ClOrdID><ClExeID></ClExeID> <Side></Side> <OrdTyp></OrdTyp> <Px>100</Px><Acct></Acct> <PriNode></PriNode> <SecNode></SecNode> <Instrmt><Sym>ABT</Sym> <ID></ID> <IDSrc></IDSrc> </Instrmt> <OrdQty><Qty>1500</Qty> </OrdQty> <meta_data></meta_data><TransactTime>05/04/2017  01:55</TransactTime> </Order>

The normalized data is fed to the normalized data channel 508. Thenormalized data can then be used by downline modules and engines such asoptimizer 510, the matching module 226, the liquidity module 228, andthe netting module 516. The normalized data is stored on the sharedpermissioned ledger 110 according to database schema. The normalizeddata is partitioned and stored on a client ledger instance based onwhich client provided the data to the data ingestion engine 224.

The normalized data channel 508 is illustrated in FIG. 5 as a singlelogical unit. However, the normalized data channel 508 may be furtherbroken down into specific channels for each client node. The purpose ofnormalized is to enable the system 100 to build models such as netting,matchers, optimizers, machine learning models, and the like on anormalized data format.

In an embodiment, each ingestor node 504 ingest data in a differentformat and in different locations. Some ingestor nodes may acquire dataavailable in, for example, an FT folder. The files may be generated inreal-time or by a batch process. Other ingestor nodes in the system 500may acquire data available on a file storage system, such as apermissioned storage system or an HDFS (Hadoop Distributed File System).Some ingestor nodes in the system 500 may acquire data available in aqueuing system such as an MQseries implementation.

Prior to ingestion, the data resides within the boundaries of theclient's systems and data centers (i.e., the client data source 402).The client must push the data to the data ingestion engine 224 for thedata to be ingested by the systems described herein. The client pushesthe data in near real-time to the data ingestion engine 224 so the datacan be ingested, normalized, and assessed in real-time to determine theclient's obligations, exposures, and real-time liquidity. The dataingestion engine 224 may include a “client push module” to establish asecure connection between the resource manager 102 and the client datasource 402 using one or more client authentication modules to push thedata from the client data source to the resource manager 102 in nearreal-time. The client may do this to handle vast amounts of data on theclient side. In an embodiment, there is no attempt to normalize messagesat the edge, and instead, the raw data is pushed to the data ingestionengine 224 in the received format. This can be important inimplementations where normalization (by the client) could alter thedata's original format in a manner that cannot be recovered oncepublished to the data ingestion engine 224. Additionally, softwareerrors in the client module could cause some data to be lost foreverprior to ingestion by the data ingestion engine 224.

The system 500 may include a node-specific event channel for each nodeassociated with a single client. In this alternative implementation (notillustrated in FIG. 5 ), each of the “client data sources” is a nodeevent channel associated with a client. One client may have a pluralityof node event channels that may push different types of data, differentvolumes of data, and data in different formats. In this case, a separateingestion event channel is opened for each of a client's nodes such thatthe system 500 dedicates a plurality of ingestion event channels to asingle client. This can resolve numerous inefficiencies and inaccuraciesthat naturally arise when ingesting enormous sums of data in nearreal-time. This enables the data ingestion engine 224 to independentlyscale processing and storage resources for each of the plurality ofingestion event channels. The volume and rate of data may be differentfor each of a client's nodes, and therefore, a single node couldoverwhelm the system 500 and add latency to data ingestion for thatclient and other clients within the system 500. This issue is overcomeby dedicating an independent ingestion event channel to each of theclient's nodes, and then scaling processing and memory resources for theindependent ingestion event channels as needed.

In an implementation where the system 500 includes a node-specificingestion event channel for each of the client's nodes, the system 500may additionally include a node-specific normalizer for each of theingestion event channels. In this implementation, the system 500includes an independent node normalizer 506 for normalizing data fromeach of a client's nodes. This can be particularly important where asingle client has multiple nodes that each push data in a differentformat. If this is the case, the system 500 can still efficientlynormalize the client's data by having a dedicated node normalizer foreach of the client's data push nodes.

Additionally, strict service-level agreements (SLAs) may be in placethat prohibit commingling to data. In some embodiments, this is treatedas a “leased line” to the client. The resource manager 102 may implementa backup policy to archive data in the shared permissioned ledger 110for longer periods of time than the client typically maintains the datain their own internal data systems. The resource manager 102 may cause acopy of the data to be stored on the shared permissioned ledger 110 inits raw format as received from the client, rather than the normalizedformat processed by the data normalizer 506.

The data received from the client data sources 402 is still in a rawformat that is custom to the client's site. The node-specific eventchannels feed into the node-specific ingestors 504 and the node-specificnormalizers 506 to normalize the data into a standard format that can beused by the resource manager 102. In an embodiment, the data is storedon the shared permissioned ledger 110 in the raw format as received fromthe client and is additionally stored in the normalized format that isorganized according to database schema. The normalized data is fed intodownline modules such as the optimizers 510, matching module 226,liquidity module 228, netting module 516, and so forth.

In an embodiment, the data normalizer 506 ingests data in the client'scustom format and maps the data to a normalized format. This can beimplemented similar to message-level ETL. The node-specific normalizerloads the data into the normalized data channel. The same designpatterns that are used for building custom Swift adapters in a clearinggateway can be used in this component as well. The platform has supportfor building a normalized format based on a pluggable architecture whichcan be used to map the custom formats to the normalized format usingtemplates. Based on the format of the customer specific data, a defaulttemplate for the format can be used and then customized for any customerspecific details. For example, SWIFT message processing can be based onthe standard SWIFT message and further enhanced to support any customerspecific fields. The platform is able to support multiple formats likeFIXML, SWIFT, XML, and comma separated files using pluggablearchitecture and templates.

The data ingestion engine 224 is a complex layer with numerous designconsiderations that compensate for the difficulty in predicting thevarious formats in which the system 500 will receive transactionmessages. In some implementations, a bank (or other financialinstitution) will use the FIX format while others will send transactiondata in internal, proprietary formats. The proprietary format mayinclude a binary format, and each asset class may have its own format.The FIX specification can define different asset classes different. Insome cases, a client will use the SWIFT messaging format for cashtransfer requests and will use different formats for other transactionrequests. Additionally, the data ingestion engine 224 consumes enormoussums of data from multiple clients in near real-time and must quicklynormalize, categorize, and store the data so that it can be processed bydownline systems.

The data ingestion engine 224 includes a parser for parsing messages andother data entries received from clients. The parser creates a set ofname-value pairs for the data contained in a message. The parser causesthe data to be nested, and the nesting is preserved when creating thename-value pairs. Once the nested name-value pairs are created, the datanormalizer 506 attempts to normalize the message into a standard format.In some cases, the system 500 can simulate SWIFT messages and normalizethose messages based on current standards.

In some cases, an exception will occur during the parsing process.Typically, the exception is a business exception but may also include aninfrastructure exception. If an exception occurs during the parsingprocess, the exception will be captured. The exception is memorializedin an exception message that is sent to an exception queue. The clientmay be granted access to the exception queue by way of a dashboard. Theexceptions are identified as messages in exception queues, some examplesare having invalid data in the message fields like alphanumeric data inamount field. Another example is having an unmatched trade. The clientsview the exception events/messages via a dashboard and can instruct toreprocess the data. Example an unmatched trade is resent for processingsince the exception could be because of delay in receipt of other sideof message.

In some implementations, as applicable, the data ingestion engine 224and the data normalizer 506 do not truncate or make any data unavailablein the normalized format. The normalized format may not use allfields/attributes of the data. Any data that is not extracted is stillmade available.

In some cases, each of the channels (e.g., the raw event channels andthe normalized data channels) have different archival policies. Anarchival policy determines the type of persistent (e.g., disk, storagearea network, HDFS, etc.) archival, and the purging of older data. Thearchival policies may be different for each node, and by association,each channel. The system 500 supports the various archival policiesthrough a configuration.

FIG. 6 is a schematic diagram of a system 600 for data ingestion. Thetrade data generator generates sample trade data required for downstreamprocesses. The trade data generator generates sample trades based oninput parameters that can be used for testing setup, different nettingstrategies, and for running liquidity simulations. This is used tosimulate where data is coming into the system from external (client)sources.

The trade data generator consumes raw data 602. The raw data 602 isinput to a raw data stream 604. The data in the raw data stream 602 isnormalized into the normalized message stream 608. The normalizedmessage stream 608 is fed to scalable file storage 606 for storage andis further fed to the matching module 226. The normalized message stream608 is additionally fed to the obligations and exposures 610 module thatcan further feed to the workflow engine 616. The results of the matchingmodule 226 and the obligations and exposures 610 modules are fed to thelookup optimized cache 612 and the dashboard 614.

The generated trade data may be generated in a suitable format such asCSV, XML, or JSON. The generated trade data includes, for example, thefollowing fields: order_id; execution_id; primary_node_id;secondary_node_id; settlement_date; settlement_cycle; symbol;order_price; fill_price; order_quantity; filled_quantity;to_be_filled_quantity.symbol: symbol of the security that is to bebought or sold; fill_price: +/−10% of the order_price. For some trades,the fill price needs to be the same as the order price; order_quantity;filled_quantity; and/or to_be_filled_quantity.

The generated trade data will have both orders and executions. An ordercan have multiple executions. For example, an order to buy 500 shares ofIBM might be executed in five separate executions, with each individualexecution having an order quantity of 100 IBM shares. An order orexecution is differentiated by the availability of the execution ID.

Orders and executions are generated for all trading hours. For example,if the data generator is triggered on 2017/05/23 with an argument togenerate 100 orders, the data generator generates 100 orders along withexecutions. Both the buy and sell side of the trade have events that areadded to the data ingestion pipeline. There is one stream for eachprimary node. So, for each trade, there will be two events (one for buyand one for the sell side of the trade). Thus, for each order execution,there will be two streams, including one data stream for the buyer andan additional data stream for the seller.

For example, suppose there are three participant nodes that are involvedin bilateral trade between themselves. The data will be generated inthree separate streams: node_a; node_b; and node_c, (corresponding tonodes 1, 2 and 3). The data in the stream node_a will be the recordswhere node1 is the primary_node_id, and so on.

The generated order will be of various types, such as completed orders,pending orders, and exceptions. If the order is completed, the datagenerator generates orders for every hour for the date on which the jobis run. While generating orders for every hour, the generator randomlycompletes an order in the subsequent hours. If the order is pending,some orders will not complete at all. If the order includes exceptions,some orders will have more executions than expected. For example, anorder of 500 shares of IBM may have six executions, with each individualexecution having an order quantity of 100 IBM shares. This is anexception because the sum of the order quantity in the individualexecutions exceeds the order quantity specified in the order.

After generating the trade samples, the data generator creates twosummary files. One summary file provides the details of the number oforders that has been generated for every hour grouped by order type.Another summary file has the cumulative count of the various types oforders at the end of every hour. For example, if the generator hasgenerated ten pending and twelve completed orders in 00 hour, and fivepending and ten completed orders in 01 hour, the summary file willinclude a count of fifteen pending and 22 completed orders for the 01hour. The generator also provides the total number of records (bothorders and executions) that have been generated in that particular run.If the size of the file in which the trade data is being generatedexceeds the maximum limit, file rotation is performed. For example, thesystem can have multiple files for a single hour such as 00.csv,00.csv.1 . . . and so on.

FIG. 7 is a schematic diagram of a process flow 700 for data ingestionand determining obligations and exposures based on the ingested data.The system includes the normalized data channel 508 wherein normalizeddata is received and partitioned as needed. The normalized data isstored on the shared permissioned ledger 110 and may be further fed tothe matching module 226. The matching module 226 identifies matchingcounterparty data entries, multiple transactions as part of a singletrade, and other data entries that should be stitched together. Thematching module 226 stitches data entries together based on metadata.

The matching module 226 identifies attributes of independent dataentries applicable to the same trade and stitches those data entriestogether to determine whether a trade has been fully settled. Thematching module 226 identifies these attributes by a(1), a(2) . . .a(n). Similar to databases, some of these attributes may be primary keysor candidate keys to uniquely identify a trade. Examples of these arecounter-party-id, cusip, trade-id, etc. The matching module 226 refersto the E(Tx Mi)=>{a1, a2 . . . an} as the events for trade ‘Tx’ ingestedfrom Mi. Likewise the systems and methods will also refer to E(Tx, Ni).A trade is said to be ‘Matched to Trade x’ when all the candidate keysfrom E(Tx Mi) match those from E(Tx, Ni). The inverse constitutes anunmatched trade. That occurs when all the candidate keys of E(Tx, Mi) donot match E(Tx Ni).

The matched data entries feed into the obligation/exposure stream 702and are stored on the exposures database 704. The obligations andexposures of each client account (and other outside parties) arecalculated based on the normalized, matched transaction data. Theobligations and exposures may be reported to the client accounts and tooutside parties by way of the obligation/exposures reporting API 706.

The matched data entries are stored on the matched orders 712 database.Order reporting and matched data entries may be provided to clientaccounts and outside parties by way of the order reporting and matchingAPI 714.

A settlement service 708 triggers a workflow 710 based on the datastored in the matched order 712 database. The settlement service is usedto execute settlement of trades as gross or netted settlement. Theservice initiates the settlement workflow between trading parties fortrades. Settlement service can initiate trades based on user input orbased on rules set agreed between parties as part of counterpartymanagement. Different states of the settlement workflow are monitored bythe workflow monitoring service and state changes published to all theobserver services like the obligation/exposure stream service. Theobligation/exposure stream services consumes incoming trades andsettlement events to calculate a node's real-time positions againstcounterparties for currency pairs.

The process flow includes a data generator to create trades for theclient nodes involved. A node-specific data ingestor consumes a specificdata stream application from the client node. A node-specific normalizernormalizes the data stream into a canonical format. The matching module226 calculates real-time order matching on the normalized data toidentify separate executions of a trade, updates to the same trade, andso forth. The matched order stream is used to calculate real-timeobligations and exposures for the parties. The obligations and exposuresare stored in the shared permissioned ledger 110 and provided to theclient nodes by way of the obligation/exposures reporting API 706. Thematched orders are stored on the shared permissioned ledger 110 to beused in other downstream processing.

The settlement service 708 and reporting layer are the final states ofthe data ingestion pipeline. These services obtain the required datafrom the analytics application. The data from the analytics applicationsis pushed to data warehouses, which are used to store a historical viewof obligations and exposures. The settlement service 708 is responsiblefor calculating the netted values of the trades that have been ingestedand for scheduling the trades to settle on the requested settlementcycle.

The workflow monitoring service 710 monitors the workflow queue tolisten for events propagated to the workflow engine. These events holdthe status of the workflow. If the status of the workflow changes, theworkflow monitoring service 710 updates the newer status into thesettlement table. Once the workflow status is COMPLETE, a feedbacksignal is sent to the matched orders stream with record type as“Feedback.”

The feedback signal is similar to the CSV data of the matched orders,which refers the buyer, seller, symbol, settlement cycle, total value,and quantity. The values of total value and quantity may be negative,which indicates the settlement has already been completed. The twofields hold aggregated values of the order value and order quantity ofthe individual orders that contribute the netting.

The data ingestion pipeline described herein introduces numerousimprovements over data ingestion systems known in the art. The dataingestion pipeline described herein is capable of consuming enormoussums of data associated with capital markets. For example, the dataingestion pipeline can handle the data throughput and latency associatedwith a large volume of financial transaction in various types of capitalmarkets. These markets can process one billion or more events on a dailybasis. Events include, for example, placing an order, updating an order,fulfilling an order, partially filling an order, initiating an order tosell, initiating an order to buy, amending an existing order, executinga transaction, and so forth. Any number of trading systems in a capitalmarket match buy orders or sell orders with one or more correspondingsell orders or buy orders. For example, an order to sell 1000 sharesinitiated by one or more trading systems. Thus, the initial order tosell 1000 shares may cause multiple buy orders, resulting in multipleevents for the single order to sell. The systems and methods describedherein can handle these types of events in high volume with acceptablelatency and in substantially real-time.

Additionally, the data ingestion pipeline described herein is useful inanalyzing the various financial data associated with the high volume ofevents. In particular, the described systems and methods can handlestreaming analytics for one or more financial markets (including allevents) across multiple financial institutions in substantially realtime.

When an order is placed with a financial institution, a liquidity demandwill be placed on the financial institution at some time in the future.The liquidity demand may indicate that the financial institution is toreceive funds in the future, or the financial institution must pay fundsin the future. The funds can be in different asset types, such ascurrency, securities, bills, bonds, and the like. Additionally, thefunds can be in different financial jurisdictions. Each liquidity demandis with a counterparty (or multiple counterparties). A particularfinancial institution with multiple liquidity demands is typicallydealing with multiple different counterparties, where each counterpartyhas different a different risk profile. Some counterparties may be moreprone to risk than other counterparties, and the risk profile of eachcounterparty may change regularly based on changing liquidity demands,changing exposures, changing obligations, credit profile, and otherfactors. The risk profile of a counterparty is also based on ajurisdiction, where some jurisdictions have a higher risk for fraud orfailed payment than other jurisdictions. For example, the risk profilefor a particular financial institution changes in substantially realtime based on what funds are owed to the financial institution and theamount of funds owed to other financial institutions.

In addition to the risk profile, a liquidity profile for a particularfinancial institution is defined based on how much is owed to thefinancial institution and how much the financial institution owes toothers (e.g., exposures and obligations). In previous systems, financialinstitutions do not have access to liquidity demands, risk profiles, orliquidity profiles for other financial institutions in substantiallyreal time. These existing systems are limited to a particular financialinstitution's data and do not provide visibility into the risk profileor liquidity profile of other financial institutions.

However, the systems and methods described herein do providesubstantially real time data associated with liquidity demandinformation for a particular financial institution by jurisdiction, byasset type, and by counterparty. These systems and methods operate in adistributed environment that includes multiple financial institutions,multiple data formats, and the like.

Additionally, the described systems and methods provide substantiallyreal time data associated with a financial institution's risk profilebased on the counterparties, jurisdictions, time zones, and the like.Thus, these systems and methods allow a particular financial institutionto determine its current liquidity profile and risk profile insubstantially real time, which was not previously available. This isaccomplished by processing streams of financial data in substantiallyreal time as it flows through the financial market. Additionally, thedata from different financial institutions and different data feeds isnormalized for consistent processing.

FIG. 8 is a schematic diagram of a system 800 for trade settlementamongst members of a settlement group. The system 100 described hereinenables trades and other financial transactions to be settled withinminutes of the trade being requested. This a significant improvementover traditional systems that require several days to determineliquidity of the parties and finalize the trade settlement. The systemsand methods described herein solve numerous technological problems inthe financial industry by moving clients to a settlement group andtracking the clients' real-time liquidity, obligations, and exposuresbased on data stored in the shared permissioned ledger 110.

Workflows, Clearing, and Settlement

The settlement of assets between participants can involve complexworkflows. The workflows may continue from the creation of settlementinstructions until assets are actually credited and debited from theaccounts. The sequence of actions for enabling a settlement can dependon the workflow and type of assets. Some commonly used settlementsequences include DVP, PVP, FOP (Free of Payments), simple one waypayments, and payments to and from segregated client accounts.

The resource manager 102 can be used to build and execute complexworkflows. Execution of the workflows results in assets being clearedand settled as agreed upon by participants. In capital markets,participants agree on a set of operating rules which include specificrules on settlement finality. The rules can differ depending on thetrade type, asset types or jurisdiction. In some cases, the rules areset by a central counterparty (CCP) or regulatory bodies. Some examplesof the operating rules are discussed in the CME US Rule Book, ICE USRule Book, ICE European Rule Book, and the Board of the InternationalOrganization of Securities Commissions on margin requirements forbilateral trades. In some embodiments, movement of assets involvesclearing and settlements across multiple banks.

In some embodiments, clearing is defined as the intra-bank movement ofassets. More specifically, clearing may involve the following steps. Onestep for clearing includes integrating into the core ledgers and paymentgateway of the financial institutions that are involved in the assetmovement. Another step for clearing includes issuing and receiving aseries of messages to determine the balance, initiating requests fordebits and credits, and getting confirmations of debits and credits. Insome embodiments, the aforementioned steps support multiple assetsclasses.

Settlement is defined as the inter-bank movement of assets in a mannerthat is final and irreversible. This may involve the following steps.One step of settlement includes integrating into the payment gateway ofthe banks to issue settlement instruction to the appropriate centralsettlement agency (e.g., US Fed or DTC in the US) to move the assetsbetween two accounts that reside at the settlement agency. Another stepof settlement includes direct communication with the central settlementagencies. In this case, participants can grant the resource manager 102a power of attorney to issue settlement instructions on behalf of theparties to perform the same settlement.

The system 800 facilitates nearly immediate trade settlement betweenclients within a settlement group. In the example illustrated in FIG. 8, the settlement system includes three clients, including client A,client B, and client C. It should be appreciated that a settlement groupmay include any number of clients, and that the efficiency of tradingwithin the settlement group will improve as additional entities join thesettlement group.

The settlement group includes a plurality of entities that agree tocommon terms for settling trades, calculating liquidity, calculatingobligations and exposures, and so forth. The clients within thesettlement group fall within the jurisdiction of the resource manager102 and agree to trade with other members of the settlement groupaccording to processes executed by the components of the system 100. Theresource manager 102 oversees and manages trades for members of thesettlement group, including the ingesting of trade data, storing tradedata on the shared permissioned ledger 110, calculating the liquidity ofmembers of the settlement group, and causing trades between members tobe executed nearly immediately. The system 100 described hereineliminates the need for time-consuming trade processes that includeverifying liquidity, transitioning funds into a suspense account, andfinally executing the transition of funds between entities.

The system 800 includes communication with external accounts ofsettlement group members 802. The external accounts may include theactual bank accounts, mutual funds, hedge funds, cash reserves,securities, and so forth of entities that have joined the settlementgroup. The external accounts include a suspense account and collateralaccount for each member of the settlement group. In the exampleillustrated in FIG. 8 , the settlement group includes client A, clientB, and client C. The external accounts 802 includes a client A suspenseaccount 804 a, a client A collateral account 806 a, a client B suspenseaccount 804 b, a client B collateral account 806 b, a client C suspenseaccount 804 c, and a client C collateral account 806 c. For settlementutility, each member bank has an account. In some embodiments, theaccounts are mirror images of the collateral accounts. Funding anddefunding refers to the act of moving money from the suspense account tothe collateral account and vice versa.

The system 800 includes the settlement system 808, which is internal tothe system 100 described herein. The settlement system 808 is overseenand managed by the resource manager 102. The settlement system includesan internal virtual account for each member of the settlement group. Inthe example illustrated in FIG. 8 , the settlement system 808 includes aclient A account 810 a, a client B account 810 b, and a client C account810 c.

The system 800 may additionally communicate with external accounts ofnon-members 812 of the suspense account. The non-member accounts maytake part in the settlement system 808 despite not having an accountwithin the bank associated with the settlement system 808. Thenon-member accounts typically work through an intermediary. Thenon-member banks may work through one or more intermediaries, andtypically there is a 1:1 relationship between the number of intermediaryaccounts associated with the number of non-member accounts. When a tradeis executed between a member of the settlement group and a non-member ofthe settlement group, the trade will require more length verificationsof liquidity to transfer assets. Nevertheless, these trades may beexecuted by communicating directly with the outside parties. Each of theoutside parties may have an associated account for crediting anddebiting trades, such as entity E account 812 e, entity F account 812 f,and entity G account 812 g illustrated in FIG. 8 .

The system 100 may ingest trade data from all settlement group members.The settlement members may be account members or non-account memberswithin the system 100. Non-account members typically work through anintermediary to settle transactions within the settlement group.

The system may execute asset transfers between banks using the ACH(Automated Clearinghouse) payment service. The ACH enables entities toelectronically collect payments from customers for either one-off orrecurring payments by directly debiting a customer's checking or savingaccounts. Common uses of ACH include online bill payment, mortgage andloan repayment, and direct deposit of payroll. Also, many investmentmanagers and brokerage firms allow users to link a bank account or anonline funding source to a trading account.

Traditionally, connecting directly to bank accounts has been preferredfor numerous reasons, including, for example: lower cost to transfermoney using ACH versus paper checks or credit cards; the ability to movelarge amounts of money; and fewer instances of fraud from ban onaccounts compared to credit cards. As discussed herein, a retail paymentis considered a movement of amounts smaller than $100,000 (although thiscan be any amount). Typically, retail payments in and out of a bankaccount are settled over settlement venues and protocols such as ACH inthe U.S., SEPA (Single Euro Payments Area), NACH in India, etc. Thesepayments have numerous advantages, including, for example: low costs;the ability to schedule automatic payments, and the ability to issue adebit pull or credit push.

Despite the aforementioned advantages of ACH, the ACH introduces somedisadvantages, including, for example: the inability to determine thevalidity of the account (this is possible if the user has closed thebank account at a later point in time); the inability to determine thebalance in the account even if valid; slow multi-phased settlementprotocol that can take hours or even days; and reject codes with theability to recall the payments later by the account holder. In somesituations, rejections in payments are in the range of 1-10% dependingon the type of products that are being purchased. For example, certaintypes of product purchases (e.g., electronics, jewelry, and the like)are more prone to fraud.

The resource manager 102 may add a funding source for moving moneybetween accounts. The resource manager 102 may add the funding source byselecting a bank from a list of retail banks using IAV (Instant AccountVerification) and/or microdeposits. IAV is done by asking a user tosubmit a username and password for the bank (or account). The processproceeds to use these credentials to log in on behalf of the user tovalidate the account. Microdeposits include a multi-step process,including the following: the user enters the account number and routingnumber of their banking institution; process makes two or more depositsof small amounts, typically less than $0.25, using the account numberand routing number; and if the above step fails, the account number androuting number are considered to be incorrect and the user has to returnto the beginning of the process to add a funding source. If there is noerror, the process proceeds to the next step. The user comes back to thewebsite or process to complete the addition of the bank account as thefunding source by validating the two micro deposit amounts. The factthat the user knew their account number and the routing number, and thenwas able to accurately validate the two micro deposits is enough proofthat the user is indeed the owner of the account. In addition, it alsosatisfies the BSA (Bank Secrecy Act) requirement for the website orprocess. When the bank account has been added as a funding source, thewebsite or process will attempt to debit money from or credit money tothe account. Debits are done when the website or process attempts to“pull” money from the account to complete a transaction. Credits aredone when the website or process allows the user to “push” money totheir bank account. This is done when the website or process has anassociated product that allows the user to hold money in their account.This can be for online payments products, brokerage accounts, taxproducts, auction sites, mortgage or rent payments, and the like.

In many systems, payments are completed over ACH or equivalent methods.The initiator is called the originator of the request. The bankingregulations require that the originator be a financial institution andis typically called the ODFI (Originating Depository FinancialInstitution) and the receiver is called the RDFI (Requesting DepositoryFinancial Institution). In the case of a debit-pull, the ODFI isrequesting a debit from the other institution. In the case of acredit-push, the ODFI pushes money to the RDFI. In most cases (but notalways), the risk is higher on the ODFI as it is the originator of therequest.

During the attempt to pull funds, there can be failures which can leadto a direct economic loss for the companies. The following is anillustrative example using an example company brokerage firm ABC-TradingInc. A customer of ABC Trading adds a bank account as a funding sourcefor their trades and allows ABC Trading to pull and push funds based ontheir trading activity with the brokerage firm. The customer instructsABC Trading to buy $5,000 worth of a stock and does not have sufficientbalance in their brokerage account to cover the purchase. ABC Tradingmakes the stock purchase and then must initiate a “pull” of $5,000 fromthe customer's bank account. ABC Trading initiates a debit-pull byissuing ACH debit instructions to its ODFI. In some cases, ABC Tradingmay be a bank and can be the ODFI. In other cases, the firm may chooseone or more banks where it has a banking relationship to originate theACH request for them. This can happen on T+0 or T+1 days depending onthe cut off time for the ODFI. The ACH debit instructions can berejected anywhere from T+0 to T+4 days. If at any point, the ACHtransfer is rejected, ABC Trading will need to undo the transaction andmay be subject to losses if the stock has lost value. There are alsooperational costs associated with tracking down the funds from thecustomer.

The steps above may be repeated many thousands of times per daydepending on the size of the broker. The process is similar for othercompanies that offer services such as bill payments, mortgage payments,or online peer-to-peer payment. The firm takes the risk of anunsuccessful debit from point T+0 to T+4 days when the request iscomplete. The rejections, despite the successful validation of theaccount, can occur due to one or more of the following. The inability tovalidate the account at point T+0: It is possible that the account mayhave been closed by the user. ABC Trading has no information about theclosure of the account at point T+0. Following the closure of theaccount, any attempt to debit the amounts from the account will resultin a rejection. Another reason for rejection is caused by insufficientbalance in the account, wherein the account did not have sufficientbalance to complete the request. In the example above, the account didnot have $5000.

FIG. 9 is a schematic diagram of a system 900 for settling tradesbetween members of a settlement group. The system 900 includes asettlement module 902. In an embodiment, the settlement module 902 is acomponent of the resource manager 102. The settlement module 902leverages information and calculations executed the matching module 226and the netting module 516. The settlement module 902 is incommunication with the shared permissioned ledger 110, a paymentdatabase 906, and a netting queue 908. The settlement module 902 managesthe nearly real-time settlement of trades between members of asettlement group. In the example illustrated in FIG. 9 , the settlementgroup includes account A, account B, account C, and account D. It shouldbe appreciated that the settlement group may include any number ofaccounts.

The settlement module 902 includes numerous adapters for communicatingwith outside parties. The settlement module 902 includes one or moresoftware modules that are trained to speak with certain dialogue ofSWIFT message. In some cases, the settlement module 902 includes asoftware module that provides instructions on how to communicate with anoutside party in a proprietary language associated with the outsideparty. For example, the settlement module 902 is configured to providelanguage-specific SWIFT messages to different banks based on theprotocols used at each bank. The settlement module 902 can be configuredto speak in a different message format for each financial institutionthe settlement module 902 communicates with. The settlement module 902additionally receives SWIFT messages from outside parties (e.g.,financial institutions such as banks, clearinghouses, hedge funds,exchanges, and so forth) and is configured to translate those messagesinto the correct format. The translated data may be recorded on theshared permissioned ledger 110. The settlement module 902 canadditionally provide a message to the counterparty to a trade indicatingthat the assets are being moved.

The system uses the matching module 226 and the netting module 516during various processing and settlement activities. A particular trademay contain a settlement cycle in addition to a value date, which allowsthe system to support multiple settlement cycles within a particularday. The settlement cycles may occur at the top of the hour. Each partywithin the settlement cycle can view incoming and outgoing dataassociated with any transaction wherein that party is a principal. Theparties cannot view data for other transactions wherein they are not aprincipal to the trade. If the party is a beneficiary of a trade, theparty cannot see the trade information but can view the settlement.

The system 100 supports multiple types of payments based on the urgencyof the settlements. For example, a particular implementation may includenormal payments, priority payments, and urgent payments. The differenttypes of payments may be thought of as priority queues.

The system 100 supports the ability for a specific payment to be splitinto multiple parts. The system 100 provides an option for a specificpayment to be split into multiple parts. The settlement module 902indicates that the smaller payments are “payparts,” or partial payments.The system 100 leverages the liquidity savings mechanisms when offeringthe ability to split up a payment. The settlement module 902 causes dataentries to be stored on the payment database 906 that link the paymentinto its payparts, and additionally pink the payparts to the parentpayment.

The liquidity savings mechanism enables numerous benefits. By default,payments are optimized to give participants the maximum liquidityefficiency. The urgent payment option is slightly different because thepayment needs to be carried out very quickly. In some implementations,the settlement module 902 will settle payments on the top of the hour.The settlement module 902 may set urgent payments to be settled everyminute. The urgent payments have the option to bypass optimization.

Once a settlement is completed, there is a feedback loop into the inputstream of the obligations and exposures processor that will adjust theobligations and exposures for the parties. In an example implementation,the trades are assumed to be bilateral such that each client node hasthe information on who it is selling to or buying from. The datageneration is currently computed at a trade ID level. The data includesone or more of the following fields: primary node, secondary node,order/trade ID, symbol, order quantity, order price, execution ID,record type, order date, and settlement date. For purposes of thisexample, three nodes are described, including node1, node2, and node3.For each order there will be a corresponding buy and sell record. Thedata is generated in three separate streams, including node_a, node_b,and node_c (corresponding with node1, node2, and node3). The data in thestream node_a will include records where node1 is the primary node. Thedata in the stream node_b will include records where node2 is theprimary node, and the data in the stream node_c will include recordswhere node3 is the primary node. The kinesis client will push the CSV(comma-separated values) records to three separate streams, includingnode-a-sample, node-b-sample, and node-c-sample.

Further to the above example, the matching module 226 assesses the buyand sell records and matches on the trade/order ID to identify thebuyer, seller, and trade information with settlementDate in a separatestream called the settlement sample. The obligations and exposuresmodule calculates the obligations and exposures on the matched orderedstream (i.e., the settlement-sample) and produces a new stream, i.e.,the obligations-exposures stream.

In some implementations, the trades need to be settled in net or ingross. One or more of the attributes of the trades E(Tx Mi)=>{a1, a2 . .. an} will indicate the settlement mode (gross vs net). The attributeswill also include other fields such as ‘value date’, ‘value amount’,‘settlement date’, ‘settlement cycle’, and the like.

The netting module 516 calculates a netting group based on one or moreof the following steps. The netting module 516 groups matched trades (asdetermined by the matching module 226) between parties based on joincriteria with an AND clause. The join criteria includes one or more ofmatching counterparty, same settlement date, same settlement cycle, andsettlement type (i.e., netted versus gross). The netting module 516calculates the sum of the value amounts for each of the groups. This isthe netted amount.

The netting module 516 calculates bilateral and multilateral netting oftransactions. Bilateral netting is a process where two parties reduce oraggregate the overall number of transactions between them. Thisbilateral netting decreases the actual transaction volume between thetwo parties. Multilateral netting is an arrangement between multipleparties that transactions be aggregated or summed rather than settledindividually. The multilateral netting process may be executed as partof a settlement group as described herein. Multilateral nettingstreamlines the settlement process among multiple parties. In somesituations, multilateral netting reduces risk by specifying that alloutstanding contracts will be terminated if there is a default or othertermination event.

The netting module 516 assigns a netting identifier to all trades withinthe netting group. The netting module 516 defines the netting ID andadditionally identifies all trade IDs that are associated with thenetting ID (i.e., all trades that will be settled as part of the nettinggroup). The netting module 516 may cause a new data entry to be storedon the shared permissioned ledger 110 that includes an indication of thenetting ID for the netting group, and further includes an indication ofall trade IDs and/or transactions that are executed as part of thenetting group. The data entry may additionally include state informationfor the netting group, for example, when the netting group was settled,whether all trades within the netting group were settled successfully,whether there was an error in settling the netting group, and so forth.

The resource manager 102 may be in communication with financialinstitutions that participate in the FX (Forex) markets. FX markets dealwith settlement of over 140 currencies. The supply and demand along withcredit extension for each of these currencies can vary significantly bycounterparty and by trading currency pair. In some implementations,these factors, along with limited visibility on payment receivables bycounterparties, may contribute to suboptimal settlement among tradingparties.

In particular situations, certain limitations may cause increased riskin the financial systems as counterparties build exposures andobligations settling occasionally. For example, problems can occur whenthere is limited overlap between settlement windows of differentcurrencies. For trades with limited overlap of currency settlements, oneside makes the payment on T+1, increasing the risk in the system.

In another example, some systems are free of payments transactions forsettling obligations, which may lead to limited visibility onfulfillment of exposures. Additionally, credit limit thresholds forcurrencies can present problems. For example, some institutions havelimited credit for various currencies and need to have their exposuresby trading parties paid before making more payments which can causedeadlocks to occur.

In another example, an end-of-day settlement can result in the mostefficient demand requirements from a treasury perspective. However, thismay lead to the greatest inter-party risk and deadlocks. This isespecially true when payments are made as FOP (Free of Payment) ratherthan PVP (Payment versus Payment), DVP (Delivery versus Payment), or DVD(Delivery versus Delivery). In some situations, gross trade settlement(PVP) has the least risk but can lead to undue strain on liquidityrequirements for the participants.

The systems and methods described herein address these possiblelimitations and problems. The system may efficiently net paymentsbilaterally and multilaterally with any number of counterparties. Thesystem may use settlement groups, which are logical groupings of partiesinvolved in various trades. These parties include principals, observers,settlement agents, and regulatory bodies. Settlement groups in thedescribed platform define a set of operational rules between parties ofthe group. Some of these rules may be related to settlement frequenciesand/or thresholds, rules related to notification of observers, andsettlement venues and/or agents for different assets.

The settlement module 902 may include suspense accounts, which arespecial purpose accounts in which account assets are protected from therisk of default by the institution at which the suspense accounts areheld. The suspense account assets are protected from default of theoriginator of the assets that fund the suspense accounts.

The netting module 516 additionally determines when trades should besettled. The netting module 516 includes one or more processors forexecuting stochastic trading liquidity models. The stochastic tradingliquidity models represent predictability models of future tradeobligations and exposures based on historical data. The stochastictrading liquidity models predict future obligations and exposures forthe parties within the settlement group. These predictions are based onnumerous factors, including one or more of: quantity and standarddeviation. The quantity includes the predictive quantity p(q) for eachasset obligation and exposure for a given counterparty.

The netting module 516 includes a demand optimization engine thatleverages output from the stochastic trading liquidity model, datastream inputs ingested by the data ingestion engine 224, data stored onthe shared permissioned ledger 110, and settlement frequencies definedas part of the settlement group. The demand optimization engineconstructs and smooths liquidity demands across settlement cycles. Thedemand optimization engine ensures the liquidity demands for settlementgroup principals closely align with end-of-trading-day requirements. Thestochastic trading liquidity models are based on historical tradingdata, such as historical trade volumes.

The data ingestion engine 224 receives data streams from the parties innear real-time. The data ingested by the data ingestion engine 224 isanalyzed by the netting module 516 and/or liquidity module 228 tocalculate overall netted obligations and exposures. The netting module516 and/or liquidity module 228 calculates the obligations and exposuresby counterparty. This is made possible by the unique structure of thesystem 100 described herein, wherein multiple parties push data to thedata ingestion engine 224 that can be assessed by the resource manager.The resource manager 102 can therefore calculate overall obligations andexposures for all parties within a settlement group falling under thejurisdiction of the resource manager 102. This significantly minimizesrisk and allows parties to execute trades within minutes, if desired.

At any given point a the liquidity module 228 is able to calculateoverall obligations and exposures by assets and counterparties. Thedemand optimization engine of the netting module 516 selects trades fora given netting cycle based on one or more of the current state ofworkflows, stochastic models built from historical trading predications,asset thresholds, and netting. The netting module 516 determines whetherthe trades within a netting cycle align with the overall bilateralnetted obligations and exposures between two counterparties with somestandard deviation for netting cycle (sn0).

Multilateral netting is an extension of bilateral demand optimization.However, in a multilateral netting cycle, the demand optimization engineoperates in two phases. In the first phase, the demand optimizationengine generates overall obligation and exposures for the cycle based onthe factors similar to bilateral netting. In the second phase, theinitial multilateral netting cycle is broken down into multiplebilateral netted cycles by the demand optimization engine. In the secondphase, the objective function of the demand optimization engine ischanged to break trades bilaterally and use the overall obligations andexposures as the thresholds between trading parties.

The netting module 516 identifies differences in the netting groupsidentified by various parties. For example, a first party in a nettingmay identify a certain set of trades to be executed in the net, and asecond party may identify a different set of trades to be executed inthe net. The netting module 516 analyzes the netting groupstrade-by-trade to identify differences in the netting groups. Intraditional systems, if the netting groups between parties are notidentical, then parties typically initiate a phone call to walk throughthe netting groups trade-by-trade and identify the differences. Thistraditional process is a time-intensive, manual process. The nettingmodule 516 described herein automates this process to determine (a) whattrades are different between the two netting groups; and (b) todetermine whether the netting group should be settled despite thedifferences based on predetermined thresholds. For example, the nettingmodule 516 may be trained to carry forward with the settlement of thenetting group if the netting calculations between counterparties are offby only a threshold amount. In this case, the counterparties may agreethat the disparate amount is negligible enough to not justify a thoroughexamination of the disparities between the netting groups. If thedisparity between the netting groups exceeds the threshold for eithercounterparty, the netting module 516 triggers a notification to allcounterparties. The counterparties may then manually review the nettinggroup and approve or deny settlement of the netting group.

The netting module 516 selects trades that will be included in a nettinggroup and settled. In most cases, the netting group includes all tradesthat have not been executed since closing of the last netting group.This includes trades left in the pipeline that have not yet beensettled. The netting module 516 groups the trades in the netting groupby generating a unique netting ID and attaching the netting ID to eachof the trades in the netting group. The netting module 516 causes a newdata entry to be stored on the shared permissioned ledger 110 for eachtrade within the netting group. The new data entry supersedes prior dataentries of that trade and indicates that the trade is part of thenetting group. The new data entry additionally provides the netting IDassociated with the trade. The shared permissioned ledger 110 cansubsequently be queried on the netting ID to identify all trades thatwere part of the netting group. Any remaining trades that are notincluded in the netting group are paused and may be recaptured in asubsequent netting group.

In some cases, the netting module 516 eliminates one or more trades thatwould normally be included in the netting group. A trade may beeliminated from a netting group because the trade does not meet athreshold defined by a client account, for example, the trade is deemedto exceed a quantity threshold, a value threshold, a risk threshold, andso forth. Client accounts can provide rules for generating nettinggroups. The netting module 516 may adhere to these rules to form customnetting groups according to client specifications. For example, thenetting module 516 may generate independent netting groups for executingonly a portion of the trades, independent netting groups based onbusiness function, independent netting groups based on asset-type,independent netting groups based on internal/external entities, and soforth.

FIG. 10 is a schematic diagram of a system and process flow forexecuting an asset transfer between client accounts and/or an externalaccount. The example illustrated in FIG. 10 includes two client accountsand one external account. The client accounts agree to the terms of asettlement group within the jurisdiction of the resource manager 102.The external account is outside the jurisdiction of the resource manager102. The client accounts and external account may include a financialinstitution such as a bank, hedge fund, and so forth.

The resource manager 102 is in communication with the (internal) clientaccount A and the (internal) client account B. As discussed herein, theresource manager 102 calculates real-time liquidity, obligations, andexposures for the client accounts based on data that is ingested fromthe client accounts in real-time and data that is stored in the sharedpermissioned ledger 110. The resource manager 102 does not havereal-time insight into the liquidity, obligations, and exposures of theexternal account. The resource manager 102 must communicate 1014directly with the external account to execute a trade between theexternal account and one or more internal accounts. The communication1014 between the resource manager 102 and the external account mayinclude SWIFT messaging. The resource manager 102 may include a softwaremodule specific to the external account that enables the resourcemanager 102 to communicate with the external account in the languageused by the external account, and further to translate messages receivedfrom the external account.

When the resource manager 102 executes a netting cycle or othersettlement with an external account (i.e., an account outside the“jurisdiction” of the resource manger 102), the resource managerreceives a data stream from the external account. The data stream may bereceived by way of a secure API, SWIFT messaging, and/or some othersecure and suitable means of communication. The resource manager 102matches the incoming data stream from the external account to anotherdata stream associated with an internal client account. The resourcemanager 102 calculates the risk profile and nets transactions based onthe matched trade data from the external account and the internalaccount.

The resource manager 102 facilitates the transfer of funds between banksassociated with client account A and client account B. In an embodiment,the suspense accounts 1012 a, 1012 b are established as part of anonboarding process when the client account joins the jurisdiction of theresource manager 102. For example, the resource manager 102administrators may work with financial institutions to establishsuspense accounts that can interact with the resource manager 102 asdescribed herein.

In some embodiments, one or more components discussed herein arecontained in a traditional infrastructure of a bank or other financialinstitution. For example, an HSM (Hardware Security Module) in a bankmay execute software or contain hardware components that interact with aresource manager 102 to facilitate the various methods and systemsdiscussed herein. In some embodiments, the HSM provides securitysignatures and other authentication mechanisms to authenticateparticipants of a transaction.

To facilitate the transfer of funds out of client A account 810 a, thefunds are transferred from client A account 810 a to the suspenseaccount A 1012 a at a bank associated with client account A 104 a. Thesuspense accounts discussed herein are “For Benefit Of” (FBO) accountsoperated by the resource manager 102 for members of the network (i.e.,client accounts under control of the resource manager 102 and otherprincipals to transactions). The resource manager 102 facilitates thetransfer of assets into and out of suspense accounts 1012. However, theresource manager 102 does not take ownership of the assets in thesuspense accounts 1012. The credits and debits associated with eachsuspense account 1012 are issued by the resource manager 102. The sharedpermissioned ledger 110 is used to track ownership of funds within thesuspense accounts 1012. Each suspense account 1012 has associatedgovernance rules that define how the suspense account 1012 operates.

At the bank associated with client account B 104 b, the transferredfunds are received by the suspense account B 1012 b. The funds are movedfrom the suspense account B 1012 b to client B account 810 b at the bankassociated with the client account B 104 b. The resource manager 102facilitates the transfer of funds between the bank associated withclient account A 104 a and the bank associated with client account B 104b.

Although only one account and one suspense account is shown for eachclient account in FIG. 10 , particular embodiments of the banksassociated with the client accounts may include any number of accountsand suspense accounts. Additionally, the banks may include any number ofinternal data systems and ledgers. The suspense accounts 1012 may beestablished as part of an onboarding process when connecting the clientaccount 104 to the resource manager 102. Resource manager 102administrators may work with financial institutions to establishsuspense accounts that can interact with the resource manager 102 asdiscussed herein.

One or more components discussed herein are contained in a traditionalinfrastructure of a bank or other financial institution, For example, anHSM (Hardware Security Module) in a bank may execute software or containhardware components that interact with the resource manager 102 tofacilitate the various methods and systems discussed herein. The HSM mayprovide security signatures and other authentication mechanisms toauthenticate participants of a transaction.

FIG. 11 is a schematic diagram of a system 1100 for executing an assettransfer between two financial institutions using two differentcurrencies. FIG. 11 specifically illustrates an example of a PVP(Payment Versus Payment) asset flow. In this example, the resourcemanager 102 is in communication with a first bank 1120 a and a secondbank 1120 b. Funds are being transferred from client A first account1122 a at the first bank 1120 to client B first account 1122 b at thesecond bank 1120 b. Funds are also transferred from client B secondaccount 1126 b through the suspense account B 1124 b and on to theclient B first account 1122 b.

The first bank 1120 a operates with USD currency and the second bank1120 b operates with GBP currency. The first bank 1120 a maintainsclient ledger instance A 112 a that identifies all transactions and dataassociated with transactions that involve the first bank 1120 a. Thesecond bank 1120 b maintains the client ledger instance B 112 b thatidentifies all transactions and data associated with transactions thatinvolve the second bank 1120 b. The client ledger instances 112 a, 112 bare maintained by the resource manager 102 as part of the sharedpermissioned ledger 110.

In the example illustrated in FIG. 11 , USD funds are being transferredout of client A first account 1122 a at the first bank 1120 a and GBPfunds are being transferred out of client B second account 1126 b at thesecond bank 1120 b. To facilitate the transfer of funds out of client Afirst account 1122 a, the funds being transferred are moved to thesuspense account A 1124 a at the first bank 1120 a. Similarly, funds aremoved from client B second account 1126 b to suspense account B 1124 bat the second bank 1120 b.

The suspense accounts discussed herein are “For Benefit Of” (FBO)accounts operated by the resource manager 102 for members of thenetwork. The resource manager 102 facilitates the transfer of assetsinto and out of the suspense accounts 1124 a, 1124 b. The resourcemanager 102 does not take ownership of the assets in the suspenseaccounts 1124 a, 1124 b. The credits and debits associated with eachsuspense account are issued by the resource manager 102 based on dataentries stored on the shared permissioned ledger 110.

At the second bank 1120 b, the transferred funds are received by thesuspense account B 1124 b. The funds are moved from the suspense accountB 1124 b to the client B first account 1122 b. Similarly, at the firstbank 1120 a, the transferred funds are received by the suspense accountA 1124 a and moved to the client A second account 1126 a. The resourcemanager 102 facilitates the transfer of funds between the first bank1120 a and the second bank 1120 b.

FIG. 12 is a schematic diagram of a system 1200 for pushing data by wayof a subscription model. The system 1200 is implemented by the resourcemanager 102 for tracking the status of different events, such ascurrency exchanges, trades, communications, and various transactions.The system 1200 includes one or more participants 1206 a, 1206 b, 1206 c(collectively referred to herein as participants 1206) in communicationwith the execution platform 106. The participants 1206 may include anyparty to an event (e.g., principals to a trade or other transaction) andmay further include client accounts managed by the resource manager 102.The participants 1206 can access data stored on their own respectiveclient ledger instances 112. The participants 1206 cannot access datastored on other client ledger instances 112, as illustrated in FIG. 12 .The participants 1206 may indicate to the data subscription 1202 modelthat they wish to be updated about state changes on an event.

The system 1200 further includes one or more observers 1204 a, 1204 b,1204 c that are not involved in a subscribed event but can neverthelessreceived updates about the event. In an example implementation, theevent includes a plurality of trades between a first principal and asecond principal. One or more of the first principal and the secondprincipal may be participants 1206 managed by the resource manager 102that can access their respective client ledger instances 112 to receiveand/or retrieve information about the plurality of trades. The observer1204 may include an outside party that needs information about theplurality of trades despite not being a part to any of the trades.Example observers 1204 include, for example, banks facilitating thetrade, exchanges, clearinghouses, regulatory agencies, and so forth. Theobserver 1204 can subscribe to one or more events (e.g., trades,communications, transactions) and receive updates about those one ormore events. The updates are retrieved from the shared permissionedledger 110 and pushed to the observer 1204 by way of the datasubscription 1202 module.

Information associated with the subscribed participants 1206 andsubscribed observers 1204 is stored in the shared permissioned ledger110. The data may be partitioned into a specific subscription database,which may be independent of the shared permissioned ledger 110 or mayinclude a portion of the shared permissioned ledger 110. The resourcemanager 102 detects state changes for a particular event and identifiesone or more transaction events that should be pushed to the subscribedparticipants 1206 and subscribed observers 1204 by way of the datasubscription 1202 module.

The participants 1206 and observers 1204 may subscribe to any number ofevent types. Example event types include workflow initiated events,clearing and settlement instruction initiated events, balance checksfailed events, overdraft requested events, overdraft approved events,and settlement completed events. When subscribing to an event type, theparticipants 1206 and observers 1204 specify the notification protocol.Example notification protocols include SMTP/email, SMS, text protocol,webhooks, push APIs, the w3c standard, and so forth. When an event typeis triggered for which there is a subscription, the appropriatenotification protocol is initiated between the resource manager 102 andthe subscriber. The subscriber is notified using the notificationprotocol in substantially real-time.

The resource manager 102 additionally includes a query interface and areporting interface to view events and associated metadata. The resourcemanager 102 provides a search API that provides the ability to searchfor the part of the shared permissioned ledger 110 that the particularsubscriber is authorized to view. Only the participants 1206 andauthorized observers 1204 to a particular event are granted access toview the details of the particular event. The search API and the reportscan further be filtered by any or all of the following categories,including asset type, date range, amount (of asset type), participantidentifier, workflow type, clearing type, and so forth.

The shared permissioned ledger 110 maintains a history of alltransactions within a network or other operating environment. The sharedpermissioned ledger 110 provides a query interface for participants andobservers to search for parts of the ledger they are authorized to view.The shared permissioned ledger 110 provides a subscription-basedinterface for participants to be notified of changes in the network asthe changes occur.

In some implementations, transactions are initiated by client accountsfor one-off money transfer requests or when a workflow is executed bythe members of a clearing group. The execution of the workflow triggersone or more transactions that reflect the movement of assets between theparticipants. Each transaction can include metadata that the principalsuse for internal business processes. Examples of the metadata includesreconciliation instructions, specific messages, and accounting code.

In some cases, payments flow as part of a Forex (FX) workflow. Forex isa market for trading currencies. In an example Forex workflow whereclient A enters into a Forex trade with client B, the followingreconciliations will be executed for the participants. Client A andclient B will choose to trade directly with the market maker or throughtheir correspondent banks that have a relationship with the marketmaker. The market maker creates the market and facilitates the trade byconnection the two parties, including one party that is buying a firstcurrency in return for one that is selling a second currency. The marketmaker earns the spread between the buying price and selling price whichmay be higher than market price. If correspondent banks are involved,the market maker wires the funds to the end accounts for clients A andB. This involves wiring funds through a central bank in respectivecountries. The market maker may have different ledger technologies inthe two countries that may operate as different legal entities. Themarket maker may have nostro accounts to reconcile the fund payments ofobligations between legal entities. A nostro account refers to anaccount at a bank that holds a foreign currency from another bank. Inthis case, multiple reconciliations may be needed, including between aclient and correspondent banks on both sides of a transaction, andbetween a correspondent bank and market makers on both sides of thetransaction.

Data is pushed by way of the data subscription 1202 module as a tradetransitions through certain events. The data subscription 1202 modelpushes trade events to participants 1206 and observers 1204 based on therules set by the participants 1206 and observers 1204. The data may bepushed in a raw format or may be pre-formatted by the resource manager102 and pushed to the participant 1206 and/or observer 1204 in a desireddata format.

The resource manager 102 implements a default data subscription 1202model for participants to a trade. The default model may be altered byany participant node. The default data subscription 1202 model generatesduplicate or semi-duplicate data entries stored on the sharedpermissioned ledger 110. If a trade includes two participants, and eachparticipant is associated with a client ledger instance 112 on theshared permissioned ledger 110, then the resource manager 102 generatesa data entry representing a state change for the trade for each clientledge instance 112. The default data subscription 1202 model can befurther enhanced by pushing data to other observer 1204 nodes.

An example data subscription 1202 includes a trigger indicating that,for example, if any trade is valued over $10 Million, then updates onthe trade will be sent to an authorized observer node associated with aregulatory body. It should be appreciated that the data subscription1202 model may include any suitable triggers as defined by clientaccounts, regulatory agencies, exchanges, clearinghouses, and so forth.

FIG. 13 is a schematic diagram of a system 1300 for pushing data eventsto participants and observers according to a subscription model. Thesystem 1300 includes a subscription database 1308, which may include aportion of the shared permissioned ledger 110, certain data entriesstored across the shared permissioned ledger 110, and/or an independentdatabase. The participants 1206 and observers 1204 to a transaction maycreate a transaction subscription 1302. The transaction subscription1302 indicates what events the subscriber wishes to be notified about,and how the subscriber wishes to be notified. The resource manager 102verifies the subscriber is authorized to subscribe to the requestedtransaction subscription 1302.

The transaction subscription 1302 is triggered by a transaction statechange 1312 and/or applicable transaction event 1310. When thetransaction subscription 1302 is triggered, the resource manager 102causes a notification to be send to the subscriber, wherein thenotification comprises information retrieved from the subscriptiondatabase 1302. The subscribers are additionally granted permission todirectly query the subscription database 1308 for information pertainingto the transaction subscription 1302.

The subscription database 1308 is an independent ledger instance on theshared permissioned database 110, similar to the independent clientledger instances 112 associated with the client accounts 104. Thesubscription database 1308 is an independent service that runs on top ofthe system 100 as part of the shared permissioned ledger 110. Thesubscription database 1308 indicates which client account nodes caninteract with other client account nodes, and further indicates whatdata can be pushed out to authorized observers 1204 and/to otherparticipants 1206 or other client account nodes within the system 100.

In an implementation, the subscription database 1308 includes areplicated copy of data stored on the shared permissioned ledger. Inanother implementation, the subscription database 1308 includes metadatapointing to data stored on the shared permissioned ledger. Thesubscription database 1308 includes data that can be pushed toparticipants 1206 and observers 1204. The data contained in thesubscription database 1308 is replicated from data stored on the sharedpermissioned ledger so that each client account knows where the datawill be replicated. Individual client accounts can see where their datais replicated and have access to any public node definitions (e.g.,regulatory bodies) so the client accounts can setup subscription rulesfor replication to non-participant nodes of the transaction.

FIG. 14 is a schematic diagram of a hash tree 1400 that can beimplemented with the methods and systems described herein for securingdata on the shared permissioned ledger 110 and other datastores. Thehash tree 1400 illustrated in FIG. 14 is a Merkle tree. A hash includesa string of numbers and letters that can be used to verify that a givenset of data is the same as the original set of transactions, but not toobtain the original set of transactions. Each transaction may be hashed,and then each pair of transactions may be concatenated and hashedtogether, and so on until there is one hash for an entire block of data.

Visualized, the hash tree 1400 resembles a tree. The hashes on thebottom row are referred to as “leaves,” the intermediate hashes arereferred to as “branches,” and the hash at the top is referred to as the“root.” The root in the example hash tree 1400 is the Hash ABCDEFGH box.The root of a given block is stored in the header and the root may becombined with other information such as the software version, theprevious block's hash, a timestamp, and so forth. The root may then runthrough a hash function to product the block's unique hash.

The transactions and metadata recorded on the shared permissioned ledger110 include information that is sensitive and confidential to thebusinesses initiating the instructions. The resource manager 102 ensuresecurity with the permissioned views and access to the sharedpermissioned ledger 110. The data for each participant is encrypted witha symmetric key that is unique for the participant. They keys may alsoinclude a key rotation policy where the data for that node is rekeyed.The keys for each node are bifurcated and saved in a secure storagelocation with role-based access controls. In some implementations, onlya special service (referred to herein as the cryptographic service 404)can access they keys at runtime to encrypt and decrypt the data.

A reconciliation report has a certain payload when generated, and thepayload, along with the public keys of the requester, are used to createa unique hash. The hash is created by hashing transaction data, forexample, principals to the transaction, amount of the transaction (inthe asset-type), account identifier, trade data, transaction data, valuedata, settlement data, and so forth. The hash is generated by applyingpublic keys of the principals to the transaction and at least one publickey associated with the resource manager 102. The has tree 1400 isgenerated for the trade bag keyset (see 1528 at FIG. 15 ) to allow forquick audits of the transactions. The hash is saved for auditingpurposes in the resource manager 102. The resource manager 102 candigitally sign the reconciliation report with a private key that makesthe reconciliation report available only to participants and authorizedobservers. The participants and authorized observers can verify theauthenticity of the reconciliation reports with an audit server (see2814 at FIG. 28 ).

FIG. 15 is a schematic diagram of a key hierarchy 1500. The multilevelkey hierarchy 1500 illustrated in FIG. 15 includes parent keys thatprovide access to all child node data and may be implemented for hashingand encrypting data stored on the shared permissioned ledger 110 andother data structures described herein. The key hierarchy 1500 includeslevel zero keys 1502, level one keys 1504, level two keys 1506, andlevel three keys 1508. The level three keys 1508 have access to levelthree data and additionally have access to level two data, level onedata, and level zero data. In some implementations, a unique documentmatching structure is created using keys generated from each of theattributes. The key hierarchy 1500 implements concepts similar to thehash tree 1400 data structure illustrated in FIG. 14 . The key hierarchy1500 is used to resolve conflicts down to an attribute-level betweenmultiple parties.

The resource manager 102 implements keys to encrypt data stored on theshared permissioned ledger 110. In some implementations, the resourcemanager 102 will only encrypt data that will be shared between partiesor transmitted. In an example, the resource manager 102 encryptsclearinghouse-level data (see 1902 at FIG. 19 ) that is provided to aclearinghouse-level authorized observer node. The resource manager 102may additionally encrypt bankruptcy-specific data (see 1904 at FIG. 19 )that is shared with other client nodes within a settlement group. Theresource manager 102 may additionally encrypt confidential data (see1906 at FIG. 19 ) that will not be shared with other parties ortransmitted but must remain confidential. In an example, a trade isrepresented by buy-side data entries and is additionally represented bysell-side data entries. A copy of each of the buy-side data entries andthe sell-side data entries may be stored on corresponding client ledgerinstances associated with counterparties to the trade. However, thebuy-side data entries will be encrypted with one key and the sell-sidedata entries will be encrypted with a different key.

The resource manager 102 includes a cryptographic service, which mayinclude a software module for encrypting data. The resource manager 102may coordinate with a third-party service for encrypting data. The keysprovide permissions and prevent unauthorized read and write access tothe data. The keys provide a means for preventing unauthorized tamperingof data.

The level three keys 1508 apply to trade bag UUIDs 1526 and data entrieswithin the trade bag keyset 1528. This level three keys 1508 mayspecifically apply to clearinghouse and exchange payment 1530 data. Thelevel two keys 1506 include the trade bundle identifiers 1522 andfurther include data entries within the trade bundle keyset 1520. Thelevel two keys 1506 may specifically apply to aggregate client payment1524 data. The level one keys 1504 apply to data entries within thetrade set UUID 1518 group. The level zero keys 1502 apply to, forexample, the CUSIP group identifier 1510, the trade data fees 1512, thetrade data 1514 itself, and the trade UUID 1516.

The trade UUID 1516, trade set UUID 1518, trade bundle UUID 1522, andtrade bag UUID 1526 each uniquely represent the trade or trade groupsinvolved in a settlement process. The associated keys provide a meansfor principals and observers to reconcile the data among theparticipants. The levels illustrated in FIG. 15 allow for participantsto only compare the data to which they have access. For instance, if aparticipant only has access to level zero keys 1502, then thatparticipant cannot validate any data at level one and above. However,users with access to a trade or trade bag at level three keys 1508 haveaccess to all levels of data, including level two, level one, and levelzero0. This permits that participant to reconcile the trades and quicklydrill down to individual trade level details to understand sources ofdiscrepancy.

The key hierarchy 1500 is used to create reconciliation reports. Thereconciliations can be created for each debit and credit pull of data.Once the documents are created in the structure illustrated in FIG. 15 ,the creation of the reconciliation report includes aggregating thedocuments for each matching party. The aggregations can be done for eachpayment cycle when assets were debited or credits to their respectiveaccounts. This is a point-of-time reconciliation. Additionally,reconciliation reports can be created over a period of time (i.e.,statements), or the reconciliation reports can be broken down by assettypes r by certain types of trade positions.

They keys are made up of hash sets according to the hash tree 1400illustrated in FIG. 14 . The keys are hierarchical and permit users todrill down to discrepancies between trading partners based on tradeidentifiers and trade information that is represented by the keys.

The resource manager 102 generates reconciliation reports according tothe key hierarchy 1500 illustrated in FIG. 15 . The reconciliationreports enable participants and authorized observers to identifyconflicts between parties. When two documents are presented, the hashesof the documents are checked against the ones previously saved. In somecases, the following steps may be executed to identify conflicts betweenthe two documents. In some implementations, the reconciliation report isnot encrypted and operates on data that is hashed using one-way hashing.In this implementation, during reconciliation, users cannot determinethe source but can compare their own hashes against a counterparty'shashes.

The conflict resolution process includes determining whether the hash ofeach party matches. If the hashes do not match, the data is not what wassent to the party by the resource manager 102. If the hashes do notmatch, the resource manager 102 attempts to identify the common set ofdocument subsets between the parties. The process walks down the levelthree keys 1508, level two keys 1506, and level one keys 1504 toidentify which ones do not match for the common set of documents. Forthe ones that do not match, the conflict resolution process continuesand walks down the list to the attribute level to identify thedifferences between the documents. Using this approach, multiple partiescan identify and reach a consensus on the correct set of attributes. Insome cases, the described systems and methods hash rows in twomismatched tables (such as in the shared permissioned ledger 110) andidentifies the specific rows that do not match. The particular columnsin the mismatched rows are analyzed to determine which specific columnsdo not match. This approach quickly finds the specific data values thatdo not match between the two tables.

The key hierarchy provides permissions to view lineage of how trades areexecuted from a business-wide perspective. Different business functions(e.g., different departments within a company) may use different keysfor encrypting data. This ensures the security of data and preventsunauthorized nodes from reading or writing on the data. In this case,the key hierarchy will include a master or parent key that has access toall node encryption keys.

The resource manager 102 implements best practices for implementingencryption keys. In some cases, the encryption keys are rotated once peryear, or once per month, or based on some other timeframe as decided bythe client or other market factors. The resource manager 102 rotates thekeys after data has been compromised. The resource manager 102 receivesthe encryption keys from a third party.

The shared permissioned ledger 110 implements bloom filters tosignificantly speed up the generation of reconciliation reports. Bloomfilters are a data structure that provide the ability to lookup if atrade is part of a chain. The bloom filter conclusively indicates if atrade is not part of the report.

FIG. 16 is a schematic diagram of an example hierarchy 1600 for groupingtrades. The hierarchy 1600 includes a trade bag keyset 1602, a tradebundle keyset 1602, and a trade set identifier 1606. The trade bundlekeyset 1602 may include a bundled collection of trade sets. The keysetsinclude hashes of trade data organized hierarchically. The keysets aregenerated based on child keysets along with data at a particular level.Trades are selected into bundles based on business rules governed bycontracts under which trades are executed. The trades are partitioned toallow for only the principals to access the trade information. Ifmultiple principals are involved, the trade data is partitioned usingkeysets which can allow for access based on permissions.

A trade bundle defines a set of properties and elements on the tradethat are encrypted using the same key. In some cases, not all elements(data points) on a trade are encrypted. This improves performance andreduces the processing resources devoted to encrypting and de-encryptingdata. The hierarchy 1600 indicates a model for which columns will beencrypted.

In an implementation, different columns of data stored on the sharedpermissioned ledger 110 are encrypted with different encryption keys.This may apply when certain columns of data store confidential orproprietary information that needs to be protected with encryption,while other columns of data do not include confidential information.Additionally, this improves security of the data by ensuring that asingle encryption key will not provide access to all columns of data.

FIG. 17 is a schematic diagram of a system 1700 for asset transferbetween settlement banks. In some cases, suspense accounts (may also bereferred to as suspense accounts) are special purpose accounts. Suspenseaccounts are protected from the risk of default of the institution atwhich the accounts are held. Suspense accounts are additionallyprotected from default of the originator of the assets that funded thesuspense account. The assets in suspense accounts are specifically forcounterparties as was specified in the settlement instruction betweenthe counterparties. Once the assets move from the sellers account intothe suspense account, the assets cannot be recalled by the seller. Thereis no unwinding of the transaction that led to the funds going into thesuspense account. In some cases, the suspense accounts may bemulti-currency and/or multi-asset accounts.

The clearing module 206 integrates to the core ledgers of banks andsettlement agencies to initiate and execute clearing and settlement.When an asset is cleared or settled, the asset moves through severalstate changes. The shared permissioned ledger 110 records the statechanges and makes that information available to the permissioned clientaccounts in real-time (or substantially real-time). Parties in a tradecan execute complex settlement instructions that determine the sequenceof steps that must be followed to affect the movement of assets betweenparticipants. The resource manager 102 facilitates this with softwareworkflows. Execution of a workflow results in multiple instructions sentand received through the clearing module 208, and further results inmultiple data entries being generated and stored on the sharedpermissioned ledger 110.

The parties of a trade can be categorized as principals, observers,settlement agents, or regulatory bodies. Principals are typically thebuyers and sellers in a transaction. For bilateral trades, this usuallyincludes the broker-dealers and buy-side clients. For cleared tradesthis may also include a clearinghouse. Observers include allintermediaries in a trade that are involved in the asset movementbetween the buyers and the sellers in the trade lifecycle. In somecases, observers may extend credits or guarantee funds to the principalsto complete a settlement.

Settlement agencies include technology companies that facilitate thepost-trade settlement of the trade. In some cases, there is more thanone settlement agent for a trade. Settlement agents typically act on theinstructions of the principals or observers. Settlement agents canadditionally provide data reports to all participants. The participantsand observers of a trade may have different regulatory reportingrequirements associated with one or more regulatory bodies. The type ofreports depend on, for example, the trade type, asset type, andjurisdiction.

FIG. 17 illustrates an embodiment of an example bilateral asset transferbetween two principals (e.g., Principal 1 and Principal 2). In thisexample, Principal 1A buys asset Y and sells asset X. Principal 2B buysasset X and sells asset Y. The resource manager 102 acts as thesettlement agent for this bilateral asset transfer. The resource manager102 utilizes suspense accounts at Settlement Bank A and Settlement BankB. In this example, both Principal 1 and Principal 2 have nostroaccounts to hold the assets of each type. A nostro accounts may includean account at a bank that holds funds in a foreign currency in anotherbank.

In the example of FIG. 17 , suspense account A and suspense account Bhold cleared funds for the benefit of a counterparty. Principal 1A is asource account for asset X (to be sold by Principal 1) and Principal 1Bis a destination account for asset Y (to be sold by Principal 2 andpurchased by Principal 1). This is also referred to as the nostroaccount for Principal 1 at Settlement Bank B.

Similarly, Principal 2A is a destination account for asset X (sold byPrincipal 1 and purchased by Principal 2). This is also referred to asthe nostro account for principal 2 at Settlement Bank A. Principal 2B isthe source account for asset Y (sold by Principal 2 and purchased byPrincipal 1).

In some embodiments, the accounts discussed with respect to FIG. 17 arelocated at different financial institutions. The resource manager 102can operate with accounts held at any financial institution. Forexample, if asset X and Y are separate currencies (USD and GBP) thenSuspense accounts that hold the USD and GBP will likely be in separatejurisdictions. In some embodiments, the existing account structures inplace at the financial institutions will not need any changing. The onlychange at the financial institutions is the creation of a new type of anaccount called the suspense account.

The resource manager 102 issues clearing and settlement instructions tothe settlement banks for each step that involves asset movement (e.g.,steps 1a, 1b, 2a, and 2b). In the case of a debit, debit the accountPrincipal 1A is debited asset X (number of units specified in thesettlement instructions which are input as parameters to the workflow)and credited to Suspense account A. Account Principal 2B is then debitedasset Y and credited to Suspense account B. Once the conformations ofboth debits are received, the system proceeds to a credit, credit whereSuspense account B is debited asset Y and Principal 1B is credited assetY. Additionally, Suspense account A is debited asset X and Principal 2Ais credited asset X.

If there is a failure in the debit, debit transactions, the resourcemanager 102 will retry the failed leg as defined in the operating rules(e.g., the operating rules may define X times to retry or define atime-based threshold). If the transaction does not succeed, it willreverse the debits and credits to restore the original amounts inaccount Principal 1A and account Principal 2B.

FIG. 18 is a schematic diagram illustrating a system 1800 forcommunication between the shared permissioned ledger 110 and outsideparties. The shared permissioned ledger stores a plurality of dataentries for each transaction, from the start of the transaction to thecompletion of the transaction. For example, the shared permissionedledger 110 stores a data entry for Start Transaction, various states inthe transaction (e.g., state one, state two, state three, and state fourillustrated in FIG. 18 ), and further includes a data entry for thecompletion of the transaction.

The shared permissioned ledger 110 provides data to participants to atransaction and also to authorized observers to the transaction. Theshared permissioned ledger 110 may provide information to an applicableclearinghouse 1802 over the transaction. The clearinghouse 1802 maycommunicate with numerous other entities and programs as illustrated inFIG. 18 . When the product is a cleared product (i.e., it has a CCP)both sides of the transaction are available to the CCP, but all of theallocation data is not available.

FIG. 19 illustrates a schematic diagram of a system 1900 for datasecurity and permissioned database access. The resource manager 102oversees data stored across the shared permissioned ledger 110 that hasvarying degrees of data privacy levels. In the example illustrated inFIG. 19 , the resource manager 102 oversees data for Principal A andPrincipal B, which are serving as counterparties to a transaction. Theresource manager 102 oversees clearinghouse-level data 1902 for each ofPrincipal A and Principal B. The resource manager 102 additionallyoversees bankruptcy-specific data 1904 for each of Principal A andPrincipal. B. The resource manager 102 ensures that confidential data1906 for each of Principal A and Principal B is not replicated.

The bankruptcy-specific data 1904 includes the obligations of eachprincipal (from the assets in its suspense accounts) to itscounterparties. This data is not shared with the CCP. This data isshared with the resource manager 102. For example, it is possible thatthis is “calculated” in real time (or substantially real time) by theresource manager 102. If Principal A is declared insolvent, then,because the assets are ring-fenced in suspense accounts, counterpartiescan rest assured that the assets will be “settled.” In this situation,settlement can mean one of the following scenarios, outlined below.

Settlement may indicate that assets are transferred to the rightfulowner (as determined by the bankruptcy specific data) but reside in thesame institution that held the suspense account. The underlyingassumption is that suspense account was held at an institution that wasnot affected by the bankruptcy of Principal A. In this situation, theresource manager 102 has the requisite information in the sharedpermissioned ledger 110 and can, upon confirmation of the bankruptcyadministrator, move the assets to Principal B.

Settlement may further indicate that assets have moved out of theexisting suspense account and institution to a different custodian orsettlement bank. In this situation, if the new custodian or settlementbank is a member of the systems and methods described herein, theresource manager 102 can, upon notification from the bankruptcyadministrator, move the assets. Alternatively, the assets can be movedby the bankruptcy administrator and the action recorded on the sharedpermissioned ledger. Because the shared permissioned ledger 110 isreplicated to Principal B, the same existing systems will get data asthey would if this event had occurred in a non-bankruptcy scenario.

Bankruptcy scenarios can be difficult situations from multiple differentperspectives, including reputational, financial stability of the overallecosystem, and trust. The transparency afforded by the sharedpermissioned ledger 110 ensures that: (a) defaulted institutionstransfer assets to their rightful owners of the assets and therebycontrol reputational risk for the ecosystem; (b) asset owners prevent a“run” on them as they never lose control or visibility of the assets;and (c) the overall ecosystem is able to control the damage stemmingfrom a failed institution.

The resource manager 102 includes reporting capabilities that followestablished norms and standards by the appropriate jurisdiction. In someembodiments, principals in a transaction can be from differentjurisdictions that may interpret failure scenarios differently and willhave different procedures to manage bank failures. For example, if aspecific jurisdiction allows leverage that exceeds United States norms,then leverage in the system outside the United States can be madeavailable to United States regulators. Although the United Statesregulators may not be able to curtail the leveraging, they will, at aminimum, be aware of the situation.

In some embodiments, the data in the shared permissioned ledger 110 isconsidered accurate and the data store (or database) is immutable. Ontop of this immutable data store is a platform that is built using stateof the art, flexible technology components. As a result, the describedsystems and methods can provide visibility that a particular regulatorseeks in a particular jurisdiction. Additionally, regulators may benodes connected to the described systems and methods. If a particularbankruptcy situation falls under the purview of a regulator that was notincluded when the systems and methods were set up, the systems andmethods can quickly provision a node for exclusive use of the particularregulator. This way, regulators can “share” data on bankruptcyscenarios. This data will follow the same accuracy and immutability asdiscussed above.

As discussed herein, the resource manager 102 may use a tieredarchitecture that can scale up to requests for clearing and settlement.This architecture provides for an automatically scaled architecturewhere specific services, such as clearing services, can scale up orshrink depending on the requests.

As discussed herein, the shared permissioned ledger 110 maintains ahistory of all transactions within the network. The shared permissionedledger 110 provides a query interface for participants to search forparts of the ledger. Additionally, it also has a subscription-basedinterface to the participants to subscribe to changes in the network inreal time (or substantially real time). In some embodiments, componentsof the shared permissioned ledger 110 include transaction states,securing the ledger entries, querying, and subscribing to the ledger,and ledger replication.

The resource manager 102 streamlines workflows by supporting richmetadata accompanying each asset transfer. The rich metadata helpsfinancial institutions tie back asset movements to trades, accounts, andclients. Payments for asset movement are reconciled between allprincipals and observers of a transaction. In some cases, reconciliationis required for internal bookkeeping of an enterprise. Additionally,certain regulations require regular filing of certain types of events.

In some implementations, payments flow between principals in a clearedmarket, such as between an end customer and a clearinghouse. Thisintroduces numerous issues with the reconciliation process in thecleared market space. For example, clearing members may act as bothbrokers and dealers to execute trades on behalf of clients and forthemselves. A clearing member typically has numerous customers, andthere are different types of trade positions that a customer mayinitiate, such as equities, futures, currency hedging, derivatives, andthe like. The clearing member will likely execute a customer's tradingactivity at more than one exchange. A customer may clear through severalclearing members.

The exchange, through a clearinghouse, may initiate settlements for alltrades executed on the exchange via the clearing members. Theclearinghouse computes net amounts that need to be debited or creditedfrom accounts of clearing members. These can be for “mark to market”variations on the trade positions. The market prices is determined bythe clearinghouse based on data from third-party sources. Followingdebits and credits to accounts, the clearing member reconciles singlenet payments to or from accounts to the positions across all clients.The clearing member sends requests for payments to each of the clients.In this step, the clearing member may add additional fees and othercharges to the payment request. The client must now reconcile these tothe actual positions. Because these are calls and may be delayed, themarket positions may change, and the market value of the trade positionmay also change. In effect, the following reconciliations need to happenbetween participants.

The clearinghouse may reconcile the net debits and credits from eachaccount at the settlement bank. In the case of a shortfall of funds, theclearinghouse requests payments from the settlement bank to authorize.The clearinghouse sends a request to the settlement bank and, whenapproved, the funds are debited. The clearinghouse additionally needs toreconcile metadata associated with the gross positions of each entitytied to the payment for each pull and push. This includes data tying tomarket data that is time bound. The fees and charges must also be tiedinto the payments. The clearinghouse must also reconcile collateralpledges and recall data tied to the payments. The payments includeadditional data attributes such as haircut amounts. The settlement ofthese assets outside the same bank will go through other settlementservices such as DTC (Depository Trust Company).

The clearing members must reconcile the net debits from their accountsthat need to be tied to the gross position for each of a plurality ofclients. Any other data such as charges and fees need to be in torequest a payment from the client or to tie in a credit push to therespective account. The clearing members additionally must reconcilepayments from client that need to be tied to specific requests fromclearing members. In some cases, the payments are not paid in full whenthere is a discrepancy between the books and data. The clearing membersmust additionally reconcile some trade positions that may not fullymatch and thus require manual adjustments at either the clearing memberor the client. Partial payments are made to fulfil obligations by eachparty and further add to the complexity in reconciliations.

Clients must reconcile net payments to and from multiple other parties.Regulators, such as the CFTC (Commodity Futures Trading Commission), SEC(Securities and Exchange Commission), ESMA (European Securities andMarkets Authority), CESR (Committee of European Securities Regulators),Federal Reserve, and the like, require different regulatory reportingfilings that tie in payment to different positions of the respectiveparties. The regulators typically request filings from multiple partiesand then run checks to ensure the records match.

FIG. 23 illustrates an example of multilateral netting calculations. Thenetting calculations can be calculated for all parties within asettlement group. The example multilateral netting calculationillustrated in FIG. 23 includes credits and debits between five partieswithin the same netting group, namely: node A, node B, node C, node D,and node E. The table illustrates various credits and debits between allnodes. For example, node A owes $250 to node B, $300 to node C, $40 tonode D, and $20 to node E. Therefore, node A has a total debit of $610to all parties within the settlement group. Each of the parties withinthe settlement group has a total credit to be received from otherparties within the settlement group, and a total debit to be paid out toother parties within the settlement group. The net position isdetermined based on the total credit and the total debit for each partywithin the settlement group.

The resource manager 102 executes the multilateral netting by creditingand debiting each party based on the net position. In the exampleillustrated in FIG. 23 , the resource manager 102 causes credits anddebits to be initiated for each party within the settlement group basedon the net positions. The resource manager 102 additionally causes newdata entries to be stored on the client ledger instances for each partyin the settlement group, wherein the new data entries reflect the netposition of the applicable party.

In some implementations, the system 100 supports independent settlementgroups for different asset types, for example, there may exist uniquesettlement groups for different currencies, securities, bonds,exchanges, commodities, and markets. The system 100 may additionallysupport settlement groups for exotic currency exchanges.

FIG. 20A is a schematic diagram of a prior art system for executingtrades between parties. As shown in the prior art system illustrated inFIG. 20A, three banks, including Bank A, Bank B, and Bank C are coupledto a central bank 2002 and an investment manager 2006. Account holders2008 are associated with the investment manager 2006. Bank A and Bank Beach have numerous accounts, and Bank C includes an investment managerreceiving account 2004 that stores funds received by or managed by theinvestment manager 2006.

The accounts may reside within the context of a bank ledger. Thisintroduces numerous issues and requires that the bank ledgers be queriedin real-time to determine if the accounts are valid and have asufficient balance. Additionally, funds must be earmarked or debited andheld in an escrow account to ensure “good funds” for a grade. Further,assets must be transferred from individual user accounts to an accountof the firm. The system 2000 illustrated in FIG. 20B represents animprovement of the prior art system illustrated in FIG. 20A.

FIG. 20B is a schematic diagram of a system 2000 for improvedtransaction execution and streamlined communications between parties.FIG. 20B represents an improvement over the prior art system illustratedin FIG. 20B. The system 2000 includes three example banks, includingBank A, Bank B, and Bank C. Bank A and Bank B each include numerousaccounts and a suspense account. The resource manager 102 is incommunication with each of Bank A and Bank B and manages the suspenseaccounts at each bank. The resource manager 102 is additionally incommunication with the investment manager 2006. Each of Bank A, Bank B,and Bank C communicates directly with the central bank 2002. Bank Cincludes an investment manager concentration account 2010, which may beused by an investment manager 2006 to facilitate the processing andsettlement of one or more customer transactions.

The arrowed lines in FIG. 20B illustrate instructions and/or themovement of assets between accounts, parties, and computer-basedsoftware constructs. The instructions include, for example, settlementinstructions associated with one or more accounts, such as Account A1,Account A2, Account A3, Account B1, Account B2, and Account B3, orsettlement accounts Suspense Account A and Suspense Account B. Theinstructions further include settlement instructions issued betweenbanks and to the central bank 2002.

Each of the accounts within Bank A and Bank B may be associated with aclient account 104 managed by the resource manager 102. One clientaccount 104 may associate multiple bank accounts with the resourcemanager 102. Each bank account may be associated with a certain type oftrade, a certain currency, a certain settlement group, and so forth.When the client accounts 104 are onboarded to the system 2000, theresource manager 102 communicates and authorizes the respective bankaccounts. The resource manager 102 controls and manages the suspenseaccounts in the banks and causes funds to be debited and creditedbetween the bank accounts and the suspense accounts. The resourcemanager causes funds to move between suspense accounts to facilitatetrades between banks accounts at Bank A and Bank B.

The system 2000 enables trades to be completed within settlement groupsat any time of day, including when banks, exchanges, and currencymarkets are closed. Members of a settlement group that are connected tothe same suspense account may move assets amongst each other during offhours, and in near real-time. A trade between members of a settlementgroup may be finalized and settled within minutes, rather than thetypical 3-5 day time period common with prior art systems as illustratedin FIG. 20A. Asset transfers from bank accounts to a suspense accountwill be executed during regular market hours, but funds within thesuspense account can be traded at any time and can be traded quickly andefficiently. The shared permissioned ledger 110 stores an indication ofthe ownership of all funds within the suspense accounts.

The suspense accounts additionally reduce risk when executing tradesbetween client accounts 104. Client accounts 104 can rest assured thattrades between other members of the settlement group will be executedsecurely, and with reduced risk, because the other party's funds aredeposited within the suspense account and overseen by the resourcemanager 102.

The suspense accounts may be associated with a certain networked groupof participants. In an example, one suspense account includes fivesettlement clients, and each of the five settlement clients is a hedgefund. In a further example, another suspense account include 200settlement clients, and each of the 200 settlement clients is venturecapitalist group. In a further example, a suspense account may include avariety of institution-types, such as hedge funds, banks, venturecapitalists, brokerages, and so forth. Typically, a suspense account isassociated with one currency. For example, a suspense account in theUnited States may trade in only USD while a suspense account in theUnited Kingdom may trade only in GBP and a suspense account in Germanymay trade only in EUR. Different suspense accounts will be held atdifferent banks, and these suspense accounts can trade with one another.In an example implementation, a singular hedge fund is associated with aclient account coupled to the resource manager 102, and this hedge fundis a member of numerous suspense accounts within different geographicregions for trading different assets and currencies.

In most cases, assets can only be transferred into and out of thesuspense accounts during operating hours for the bank where the suspenseaccount is located. However, funds can be traded amongst settlementclients associated with the suspense account during any hours. Thesetrades are tracked on the shared permissioned ledger 110 and can beexecuted in near real-time. Additionally, the suspense accounts reducerisk for all settlement clients associated with the suspense accounts,because the resource manager 102 can continuously track the obligationsand exposures for the settlement clients associated with the suspenseaccount.

The suspense accounts are asset-agnostic and can be associated withtrades of different currencies, securities, bonds, Forex, and so forth.One suspense account may hold funds associated with a plurality ofsettlement clients. Each of the settlement clients associated with thesuspense account can trade assets amongst each other more easily andwith reduced risk.

Transactions between members of a suspense account (i.e., the settlementclients associated with the suspense account) does not require actualmovement of funds. However, the movement of assets into and out ofsuspense accounts, and between different suspense accounts, requires theactual movement of funds. These movements will be reported to thecentral bank 2002. The shared permissioned ledger 110 serves as acentralized ledger for each member of a settlement group for all assetswithin the suspense account. The investment manager 2006 managestransfers of funds from suspense accounts to other accounts external tothe system 100. Funds can be moved to the investment managerconcentration account 2010 in some implementations, but this is notrequired to realize the benefits of the suspense accounts.

FIG. 21 is a schematic diagram of a system 2100 for communicationsbetween computing nodes and a distributed transaction store. As shown inFIG. 21 , two nodes, including Node A 2102 and Node B 2104 are coupledto the shared permissioned ledger 110. The nodes may representorganizations, companies, participants, regulators, observers,individuals, and so forth. The shared permissioned ledger 110 storesvarious transaction-related data associated with any number of nodes, aswell as transaction data associated with other entities or systems.

The onboarding process involves registering a new node (e.g., Node A2102 or Node B 2104) with the resource manager 102. In some embodiments,a node is a virtual entity that corresponds to a legal entity in thereal world. In addition to common metadata such as a name, type (direct,indirect, observer, and the like), and the like, the node setup involvesproviding a globally unique identifier for the node. This uniqueidentifier can be used to refer to the node across the resource manager102. A node can be searched in a directory either by using its name orits unique identifier. The setup process involves installing digitalcertificates that can identify the node in the financial managementecosystem as well as among its participants.

Authentication of participants and their transactions is achieved with aPublic Key Infrastructure. Each participant will set up an Adminidentity to control issuance of certificates to its users. The users canenroll with their organization's Certificate Authority and use theirprivate key and certificate to sign transactions and identifythemselves.

While a node is a virtual entity configured in the resource manager 102ecosystem, the node can have its own legacy or current system backed bya data store 2106, 2108 to carry out its process. In some embodiments,integration to these systems are carried out by “adaptors” that areassociated with the node. For example, adaptors can be software and/orhardware modules that are typically custom-developed, installed, andconfigured as part of the node. In particular implementations, theconfiguration process involves capturing a protocol of communication,capturing connectivity details that are specific to the protocol, andsetting up identities. An important step, discussed herein, is thesetting up of identities. This may involve importing or re-configuringthe identity details such as certificates that are required to identifythe node and its users in the existing legacy system. For example, thiscan involve the signatures used to sign the transactions in theirinternal DLT (Distributed Ledger Technology).

As shown in FIG. 21 , the custom adaptor modules can follow either apush model or a pull model. In the push model, the adaptor is listeningfor updates happening in the underlying system and the same getspropagated into the financial management ecosystem discussed herein. Inthe pull model, the adaptor pulls the necessary details and pushes theminto the node. In addition to transactions, the adaptor can also listento and raise events that are relevant to the process being automated.Thus, the adaptor is both a publisher as well as a subscriber for theevents in the underlying service.

FIG. 22 illustrates an example of bilateral netting calculations. Thenetting module 516 described herein calculates netting positions forparties by assessing an incoming normalized data stream in real-time.The netting module 516 can additionally access data stored on the shareddatabase ledger 110 to calculate netting positions.

The example bilateral netting calculation illustrated in FIG. 22includes various credits and debits from account A against account B,and additionally includes credits and debits from account B againstaccount A. The instruction net value from A to B is $800 such that Aowes $800 to B. The instruction net value from B to A is $700 such thatB owes $700 to A. the bilateral net position is a net credit of $100from A to B.

FIG. 23 illustrates an example of multilateral netting calculations. Thenetting calculations can be calculated for all parties within asettlement group. The example multilateral netting calculationillustrated in FIG. 23 includes credits and debits between five partieswithin the same netting group, namely: node A, node B, node C, node D,and node E. The table illustrates various credits and debits between allnodes. For example, node A owes $250 to node B, $300 to node C, $40 tonode D, and $20 to node E. Therefore, node A has a total debit of $610to all parties within the settlement group. Each of the parties withinthe settlement group has a total credit to be received from otherparties within the settlement group, and a total debit to be paid out toother parties within the settlement group. The net position isdetermined based on the total credit and the total debit for each partywithin the settlement group.

The resource manager 102 executes the multilateral netting by creditingand debiting each party based on the net position. In the exampleillustrated in FIG. 23 , the resource manager 102 causes credits anddebits to be initiated for each party within the settlement group basedon the net positions. The resource manager 102 additionally causes newdata entries to be stored on the client ledger instances for each partyin the settlement group, wherein the new data entries reflect the netposition of the applicable party.

In some implementations, the system 100 supports independent settlementgroups for different asset types, for example, there may exist uniquesettlement groups for different currencies, securities, bonds,exchanges, commodities, and markets. The system 100 may additionallysupport settlement groups for exotic currency exchanges.

FIG. 24 is a schematic diagram of a settlement group 2402 as describedherein. The systems and methods described herein deploy settlementgroups comprising a plurality of client accounts. All parties within asettlement group must agree to the terms and conditions of thesettlement group, including, for example, how liquidity, obligations,and demands are calculated for parties within the settlement group,thresholds for netting, thresholds for settling, and so forth. Theparties in the settlement group may request additional thresholds forexecuting netting cycles and settling trades between other parties.

The settlement group 2402 described herein enables numerous benefitsover traditional settlement systems. Parties within the settlement group2402 fall under the jurisdiction of the resource manager 102 such thatthe resource manager has insight into the real-time trade dataassociated with the parties and can therefore calculate the liquidity ofthe parties in real-time. Parties within the settlement group 2402 cancomplete trades with each other nearly immediately after a trade isrequested. Additionally, parties within the settlement group 2402 cancomplete trades with each other after-hours when institutions such asbanks, clearinghouses, and exchanges are closed. These significantimprovements are enabled by the computer-centric improvements describedherein, wherein the client accounts within the settlement group aremanaged by a resource manger 102 that can oversee incoming data streamsand calculate real-time liquidity, obligations, and exposures based ondata stored within the client ledger instances 112.

The example settlement group 2402 illustrated in FIG. 24 includes theresource manager 102 in communication with a plurality of clientaccounts that have opted-in to the settlement group 2402. The clientaccounts included in the example include client account A 104 a throughclient account H 104 h. It should be appreciated that the settlementgroup 2402 may include any number of client accounts and/or independentnodes associated with single entities.

It should be noted that a client account can operate under thejurisdiction of the resource manager 102 without opting into anysettlement group. However, the maximum benefits of the system 100 arerealized when client accounts opt into settlement groups. In anembodiment, independent settlement groups are established for tradingdifferent assets. For example, independent settlement groups may beestablished for different currencies, for bonds, securities, and soforth. A client account can join any number of settlement groups and maychoose to join a settlement group for each asset the client accounttrades with.

The resource manager 102 can manage trades between the client accountswithin the settlement group 2402 in real-time such that trades can beexecuted after-hours and nearly immediately after the trade isrequested. The resource manager 102 includes a multilateral nettingpositions 2404 module and a suspense account 2406 module for overseeingand managing operations of the settlement group 2402.

Settling is the act of closing out obligations between principals of atrade. The settlement is the act that involves the movement of assets.In some cases, the parties of the trade agree on a point in time in thefuture to settle the trades. This determination can be based on acertain time frame (e.g., settle one-hour before markets close), meetinga threshold number of trades or asset value, meeting a thresholdliquidity, and so forth. Not all trades will be settled, and some typesof trades are never settled and just roll forward. The principals maydecide to run multiple settlement cycles on a settlement date.

The multilateral netting positions 2404 module calculates multilateralnetting positions between the client accounts within the settlementgroup. The multilateral netting positions 2404 module may additionallycalculate bilateral netting positions between any two parties within thesettlement group 2402 or outside the settlement group 2402. Themultilateral netting positions 2404 module calculates dynamic nettingpositions that are adjusted in near real-time as trade data is consumedby the data ingestion engine 224.

The suspense account 2406 The shared permissioned ledger tracks thebalances each participant has in the suspense account. The ownershipchange for funds in the suspense account can happen independent of thesettlement rails or hours of operations. Only transfer of funds outsideof settlement bank requires the currency specific settlement times.Internal bank transfers can happen independent of the settlementcutoffs. The system 100 manages the suspense account 2406, which may belocated at a third-party bank or other institution but is managed by thesystem 100 for the benefit of all participants within the settlementgroup 2402.

FIG. 25 illustrates example data entries stored on the sharedpermissioned ledger 110 for counterparties to a trade. In the exampleillustrated in FIG. 25 , client account A initiates a trade with clientaccount B. The client ledger instance A 112 a associated with clientaccount A stores a ledger entry indicating that a new trade has beeninitiated by client account A, and that B is a counterparty. The dataentry includes the unique trade identifier associated with the trade. Acorresponding data entry is stored on client ledger instance B 112 bassociated with client account B. The corresponding data entry indicatesthat a new trade has been initiated with client account A, wherein A andB are counterparties. This data entry also includes the unique tradeidentifier assigned to the trade.

The client ledger instances 112 additionally include independent dataentries for updates to the workflow for executing the trade. Each ofthese independent data entries includes unique trade identifierassociated with the trade. These additional data entries may indicate,for example, that funds have been moved to a suspense account, that theworkflow has determined whether sufficient funds are available in thesuspense account, whether the trade has been approved by either party,and so forth. When the trade is settled, each of the client ledgerinstances 112 will store a data entry indicating that the trade has beensettled. The data entry stored on client ledger instance A 112 a willindicate that a debit on account A has been executed. The data entrystored on client ledger instance B 112 b will indicate that a credit onaccount B has been executed. These data entries will also include theunique trade identifier associated with the trade.

The states stored in the ledger are governed by the states representedin the smart workflow governing the settlement process. Based ontriggering events and the participants of the trade, the settlementservice chooses the workflow to settle the funds. The workflows for aclearing group are agreed to as part of onboarding to a clearing group.The choice of workflow determines the steps/states the workflow willhave. All states of the workflow are stored in the ledger. The workflowhas 2 types of states, public states which are shared between theparticipants and private states which correspond to only a particularparticipant. The public states will be stored in both ledgers, howeveronly the principal might have access to the underlying details of thetransaction corresponding to the state. For example, the receipt of thefunds could be a public state corresponding to a SWIFT message. Thedetails of the SWIFT message will only be visible to the principal, andthe other participant will only see an acknowledgement that the workflowstate has changed due to receipt of the funds. The private states areonly stored in client specific ledger, these correspond to any localsteps which the transaction goes through which are not shared with otherparticipants. An example could be an approval step for transfer of fundsover a certain threshold.

The shared permissioned ledger 110 implements various constructs toensure data integrity. The shared permissioned ledger 110 includescryptographic safeguards that allow a transaction to span 1−nprincipals. The resource manager 102 ensures that no other users (otherthan the principals who are parties to the transaction) can view data intransit. Additionally, no other user should have visibility into thedata as it traverses the various channels. In some embodiments, there isa confirmation that a transaction was received completely and correctly.The resource manager 102 also handles failure scenarios, such as loss ofconnectivity in the middle of the transaction. Any data transmitted to asystem or device is explicitly authorized such that each data entry onthe shared permissioned ledger 110 can only be seen and read by theprincipals who were a party to the transaction. Additionally, principalscan give permission to regulators and other individuals to view the dataselectively.

In some embodiments, the resource manager monitors for data tampering.If the data store (central data store or replicated data store) iscompromised in any way and the data is altered, the resource managershould be able to detect exactly what changed. Specifically, theresource manager should guarantee all participants on the network thattheir data has not been compromised or changed. Information associatedwith changes are made available via events such that the events can besent to principals via messaging or available to view on, for example, auser interface. Regarding data forensics, the resource manager is ableto determine that the previous value of an attribute was X, it is now Yand it was changed at time T, by a person A. If a system is hacked orcompromised, there may be any number of changes to attribute X and allof those changes are captured by the resource manager, which makes thetampering evident.

In particular embodiments, the resource manager leverages the bestsecurity practices for SaaS (Software as a Service) platforms to providecryptographic safeguards for ensuring integrity of the data. Forensuring data integrity, the handshake between the client and an APIserver 2812 (discussed with respect to FIG. 6 ) establish a mechanismwhich allows both the client and the server to verify the authenticityof transactions independently. Additionally, the handshake provides amechanism for both the client and the server to agree on a state of theledger. If a disagreement occurs, the ledger can be queried to determinethe source of the conflict.

The shared permissioned ledger 110 records states changes for atransaction, and in some cases, a transaction is rolled back due to anunrecoverable error, a cancellation of the transaction, and so forth.Additionally, the shared permissioned ledger 110 records when atransaction is “clearing pending,” “cleared,” “settlement pending,”“completed,” and/or “cancelled.” Each transaction and the associatedtransaction states include additional metadata. The shared permissionedledger 110 stores the state information and state changes for thetransaction. A separate record is maintained for each state of thetransaction. The records cannot be updated or modified.

In some cases, a transaction is created in error (e.g., “fat finger”).For these erroneous transactions to be reversed, the shared permissionedledger 110 records a new transaction. The metadata for the newtransaction includes a reference to the erroneous transaction that needsto be reversed. The parties are informed of the request to reverse theerroneous transaction as part of a new transaction. The new transactionalso goes through the state changes illustrated in FIG. 29 . When thenew transaction is completed, the metadata of the initial transaction isalso updated.

FIG. 26 is a schematic diagram of a system and process flow for matchingtrades and determining whether a trade is fully settled. The systemincludes a client ledger instance 112 of the shared permissioned ledgerand a client processing node 108 of the execution platform 106. Theprocessing node 108 executes instances of the resource manager 102 forthe client account associated with the client ledger instance 112.

The client ledger instance 112 includes a plurality of data entriesapplicable to a single trade. The trade has been executed in a pluralityof trade-lets, including a first transaction, a second transaction, anda third transaction. The client ledger instance 112 includes at leastone independent data entry for each of the trade-let executions of thetrade. The client ledger instance 112 includes a state of the firsttransaction of the trade 2604, and this data entry includes a hashedtrade ID associated with the trade. The client ledger instance 112similarly includes a state of the second transaction of the trade 2606that also includes the hashed trade ID associated with the trade; and astate of the third transaction of the trade 2608 which also includes ahashed trade ID associated with the trade.

The processing node 108 determines whether the trade has been settled infull. The processing node 108 executes a transaction associated with thetrade ID at 2610. The processing node 108 queries the client ledgerinstance for all transaction associated with the trade ID at 2612. Theprocessing node 108 determines whether the trade is fully settled basedon all ledger entries comprising the trade ID at 2614. In response todetermining that the trade has been fully settled, the processing node108 generates a new ledger entry indicating that the trade is COMPLETEDat 2616. Alternatively, the processing node 108 may determine that thetrade has been partially settled. The processing node 108 may furtherdetermine whether an EXCEPTION has occurred on the trade because thetrade has been over-settled beyond the initial order amount.

FIG. 27 is a schematic diagram of a system and process flow fornegotiating a trade split between counterparties. The system includes asettlement group 2402. The settlement group includes one or more clientaccounts, outside entities, and/or observers. In the example illustratedin FIG. 27 , the settlement group 2402 includes at least three clientaccounts 104. Each of the client accounts includes a processing node 108and a trade split module 2604 that is executed by the processing node108.

A process flow begins with a party requesting a trade. The partyrequesting the trade may be within the settlement group 2402 or outsidethe settlement group 2402. In an example implementation, the partyrequesting the trade is part of the settlement group 2402 and thecounterparty to the trade is also part of the settlement group 2402. Theprocessing node associated with a counterparty to a trade (i.e., theparty requesting the trade and/or another party) suggests a trade splitat 2606. The trade split is determined based on one or more factorsincluding, for example, the overall obligations and exposures of thecounterparties; the current liquidity of the counterparties; thepermissible lot size; thresholds defined by the settlement group 2402and/or the counterparties; the number of settlement cycles left in acertain time period; a number of pending orders to be completed by thecounterparties and/or the settlement group 2402; a number of orders leftto be completed in the day, and so forth. The processing node 108 thatinitially suggests the trade split at 2606 does not have access to allinformation associated with the trade, and specifically does not haveaccess to information indicating the liquidity, obligations, andexposures of the counterparty to the trade. The processing node 108 mustmake these calculations based on the outputs from predictive models,information stored in the client ledger instance 112 associated withthat processing node 108, and other data that is available to theprocessing node 108.

The processing node presents the potential trade split to the one ormore counterparties. The one or more counterparties approve or declinethe trade split at 2608. If the one or more counterparties approve thetrade split at 2608, the trade will be executed according to the tradesplit at 2610. If the one or more counterparties decline the trade splitat 2608, automated negotiations will begin between the applicableprocessing nodes at 2612 to determine the trade split. The negotiationsmay continue until the processing nodes (or users associated with theapplicable accounts) agree to a trade split. If a trade split cannot beagreed upon, the trade may be cancelled.

The trade split indicates how the trade will be executed over time. Forexample, a trade may include an order to exchange 10 million USD for 8million EUR. The trade may be executed through the foreign currencyexchange market. The parties might not have the required liquidityon-hand to complete the trade in a single execution. The trade may beexecuted over a series of trade-let executions. For example, the trademay be settled with four separate trade-let executions, each for 2.5million USD and 2 million EUR. The trade-lets may be executed atdifferent times throughout a single day or may be executed over thecourse of several days. The parties to the trade agree to a trade splitprior to the trade being settled.

FIG. 28 is a schematic diagram of a system 2800 for centralizedfacilitation of financial transactions between unrelated parties. Thesystem includes the resource manager 102, the network 118, and theshared permissioned ledger 110 as described herein. In an implementationwherein the resource manager 102 oversees and facilitates financialtransactions between parties, the resource manger 102 may communicatewith financial institutions 2806, authorized systems 2808, andauthorized user devices 2810 by way of the network 118 connection. Theresource manager 102 additionally communicates with a replicated data2804 database. The resource manager 102 includes and/or communicateswith an API server 2812 and an audit server 2814.

In an embodiment, all interactions with the resource manager 102 and/orAPI server 2912 are secured with TLS (Transport Layer Security). The APIserver 2812 and the audit server 2814 communicate with the resourcemanager 102 by way of a suitable data communication link or datacommunication network, such as the network 118 illustrated in FIG. 28 .The API server 2812 and the audit server 2814 may be incorporated in theresource manager 102. In a particular implementation, a single serverperforms the functions of the API server 2812 and the audit server 2814.

The replicated data 2804 stores data accessible to authorized systemsand devices. The replicated data 2804 is a database storing immutableand auditable forms of transaction data between financial institutions.The immutable data cannot be deleted or modified and can instead only bereplaced with new data entries referencing the outdated entries. Thereplicated data 2804 database may store append-only data that keepstrack of all intermediate states of the transactions. Additionalmetadata may be stored along with the transaction data for referencinginformation available in external systems. In an embodiment, thereplicated data 2804 datastore provides read-access to outside,authorized parties to read information associated with clients 104 incommunication with the resource manager 102.

The resource manager 102 may communicate with outside parties by way ofsecure APIs. The API server 2812 of the resource manager 102 is securedwith TLS (transport layer security). The resource manager 102 mayadditionally communicate with an audit server 2814. The API server 2812and the audit server 2814 communicate with the resource manager 102using a suitable data communication link or data communication network,such as a local area network or the Internet. The API server 2812 andthe audit server 2814 may be incorporated into the resource manager 102.

In some embodiments, at startup, a client sends a few checksums it hassent and transaction IDs to the API server 2812, which can verify thechecksums and transaction IDs, and take additional traffic from theclient upon verification. In the case of a new client, mutually agreedupon seed data is used at startup. A client request may be accompaniedby a client signature and, in some cases, a previous signature sent bythe server. The API server 2812 verifies the client request and theprevious server signature to acknowledge the client request. The clientpersists the last server signature and a random set of server hashes forauditing. Both client and server signatures are saved with requests tohelp quickly audit correctness of the resource manager ledger. The blocksize of transactions contained in the request may be determined by theclient. A client SDK (Software Development Kit) assists with the clientserver handshake and embedding on server side signatures. The SDK alsopersists a configurable amount of server signatures to help with restartand for random audits. Clients can also set appropriate block size forrequests depending on their transaction rates. The embedding of previousserver signatures in the current client block provides a way to chainrequests and provide an easy mechanism to detect tampering. In additionto a client-side signature, the requests are encrypted using public keycryptography to provide additional defense against client impersonation.The API server 2812 logs encrypted requests from the client. Theencrypted requests are used, for example, during data forensics toresolve any disputes.

A client may communicate a combination of a previous checksum, a currenttransaction, and a hash of the current transaction to the resourcemanager 102. Upon receipt of the information, the resource manager 102checks the previous checksum and computes a new checksum, and stores theclient hash, the current transaction, and the current checksum in astorage device, such as the system data 116 database. The checksumhistory and hash the integrity of the data. Any modification to anexisting row in the shared permissioned ledger 110 cannot be made easilybecause it would be detected by mismatched checksums in the historicaldata, thereby making it difficult to alter the data.

The integrity of the resource manager 102 is ensured by having serveraudits at regular intervals. The resource manager 102 uses chainedsignatures per client at the resource manager, and this ensures that anadministrator of resource manager 102 cannot delete or update anyentries without making the shared permissioned ledger 110 tamperevident. In some embodiments, the auditing is done at two levels: aminimal level which the SDK enforces using a randomly selected set ofserver signatures to perform an audit check; and a more thorough auditcheck run at less frequent intervals to ensure that the data is correct.

The resource manager 102 allows for the selective replication of data.The replicated data can be provided to the replicated data 2804database, and these transactions can be made available to authorizedusers, including principal parties to a transaction and observers to thetransaction. This approach allows principals or banks to only hold datafor transactions they were a party to, while avoiding storage of otherdata related to transactions in which they were not involved.Additionally, resource manager 102 does not require clients to maintaina copy of the data associated with their transactions. Clients canrequest the data to be replicated to them at any time. Clients canverify the authenticity of the data stored on the shared permissionedledger 110 by using the replicated data and comparing the signature theclient sent to the resource manager with the request.

The resource manager 102 may communicate with a notarial system tomaintain auditability and forensics for the core systems. Rather thanrelying on a single notary hosted by the resource manager 102,particular embodiments allow the notarial system to be installed andexecuted on any system that interacts with the resource manager 102(e.g., financial institutions or clients that facilitate transactionsinitiated by the resource manager).

The resource manager 102 leverages the best security practices for SaaS(Software as a Service) platforms to provide cryptographic safeguardsfor ensuring integrity of the data. To ensure data integrity, thehandshake between the client account and an API server establish amechanism which allows both the client and the server to verify theauthenticity of transaction independently. The handshake provides amechanism for both the client account and the server to agree on a stateof the shared permissioned ledger. If a disagreement occurs, the sharedpermissioned ledger is queried to determine the source of the conflict.

The systems and methods discussed herein support different assetclasses. Each asset class may have a supporting set of metadatacharacteristics that are distinct. Additionally, the requests and datamay be communicated through multiple “hops” between the originatingsystem and the resource manager. During these hops, data may beaugmented (e.g., adding trade positions, account details, and the like)or changed. In certain types of transactions, such as cash transactions,the resource manager 102 streamlines the workflow by supporting richmetadata accompanying each cash transfer. This rich metadata helps bankstie back cash movements to trades, accounts, and clients.

FIG. 29 illustrates an example state diagram 2900 showing various statesthat a transaction may pass through. As shown in FIG. 29 , a particulartransaction may be initiated (“new”), then clearing is initiated with abank, after which the transaction's state is “clearing pending.” Thenext transaction state is “cleared”, then settlement is initiated, afterwhich the transaction state is “settlement pending.” After thetransaction has settled, the state becomes “completed.” As shown instate diagram 2900, the state diagram may branch to “cancelled” atlocations in the state diagram. For example, a transaction may becancelled due to insufficient funds, a mutual decision to reverse thetransaction before settlement, a bank internal ledger failure, and thelike. Additionally, the state diagram may branch to “rolled back” atmultiple locations. For example, a transaction may be rolled back due toan unrecoverable error, a cancellation of the transaction, and the like.

Each transaction and the associated transaction states may haveadditional metadata. The shared permissioned ledger 110 may contain thestate information and state changes for a transaction. A separate recordis maintained for each state of the transaction. The record is notupdated or modified. In some embodiments, all transactions are final andirreversible. The metadata for the new transaction includes a referenceto the erroneous transaction that needs to be reversed. The parties areinformed of the request to reverse the erroneous transaction as part ofa new transaction. The new transaction also goes through the statechanges shown in FIG. 29 . When the new transaction is completed, themetadata of the initial transaction is also updated.

In some embodiments, the transactions and the metadata recorded in theshared permissioned ledger contain information that are very sensitiveand confidential to the businesses initiating the instructions. Thesystems and methods described herein maintain the security of thisinformation by encrypting data for each participant using a symmetrickey that is unique to the participant. In some embodiments, the keysalso have a key rotation policy where the data for that node is rekeyed.The keys for each node are bifurcated and saved in a secure storagelocation with role-based access controls. In some embodiments, only aspecial service called a cryptographic service can access these keys atruntime to encrypt and decrypt the data.

FIG. 30 is a schematic flow chart diagram of a method 3000 fortransferring assets (e.g., funds) between two banks in differentcurrencies. The method may represent an execution of the asset transferillustrated in FIG. 11 . The method 3000 begins and the resource manager102 receives at 3002 a request to transfer funds from an account at afirst bank in a first currency to an account at a second bank in asecond currency. The resource manager 102 confirms at 3004 availablefunds for the transfer in each of the first currency at the first bankand the second currency at the second bank. The resource manager 102causes at 3006 an account at the first bank to be debited by thetransfer amount and a first suspense account at the first bank to becredited by the transfer amount in the first currency. The resourcemanager 102 causes at 3008 an account at the second bank to be debitedby the transfer amount and a second suspense account at the second bankto be credited by the transfer amount in the second currency. Thetransferred funds are settled at 3010 from the first suspense account tothe second suspense account, and additionally from the second suspenseaccount to the first suspense account. The resource manager causes at3012 the first suspense account to be debited by the transfer amount andcredited in the first currency to the account at the first bank, andfurther causes the second suspense account to be debited by the transferamount and credited in the second currency to the account at the secondbank.

FIG. 31 is a schematic flow chart diagram of a method 3100 forinitiating a workflow and generating reconciliation data applicable tothe workflow. Initially, one or more principals to a transaction areonboarded with the resource manager 102. The principals associate theirrespective client accounts with real-world banking accounts held atvarious banks or other financial institutions. These associated bankingaccounts are used for debiting and crediting funds.

The method 3100 begins and the resource manager 102 receives at 3102 arequest to initiate a workflow for a transaction from one or moreprincipals to the transaction. The resource manager 102 receives at 3104metadata from the one or more principals to the transaction thatassociates payment debits and credits with the transaction. The resourcemanager 102 generates at 3106 one or more new data entries to be storedon a shared permissioned ledger in response to identifying one or morestate changes on the transaction. The resource manager 102 receives at3108 additional metadata from the one or more principals to thetransaction that identifies payments to clients of the one or moreprincipals. The resource manager 102 identifies at 3110 reconciliationdata for the transaction and communicates the reconciliation data to theone or more principals.

The resource manager 102 orchestrates the systems and methods discussedherein into a workflow. The workflow may be initiated by authorizedusers and/or may be automatically triggered by the resource manager 102.When a payment is initiated, the appropriate metadata is passed, andwhen possible, the resource manager 102 uses the ISO 20022 format forpassing additional metadata. The metadata is structured and tiespayments in with the amounts to be debited and credited to eachprincipal, and the type of payments. The type of payments includes, forexample, whether the payment includes a net payment or gross payment. Ifthe payment is net, the metadata includes the makeup of each uniquegroup of like fees, margins, and so forth. If some of these types have asub-type, the metadata includes the appropriate subtype (e.g., initialmargin versus variation margin, and so forth). For each group of items,the metadata includes the gross position by each trade position. Foreach trade position, additional metadata may be passed. For example, theCUSIP (Committee on Uniform Security Identification Procedures) number,the number of units, the market prices, fees, and so forth.

Each principal can utilize multiple accounts using the systems andmethods described herein. The resource manager 102 sends each bank thatholds each account the appropriate messages to initiate the debits andcredits. If any of the payments require a settlement message to be sent,the resource manager 102 initiates the appropriate settlement messagewith the central bank. The shared permissioned ledger 110 records allfor the transactions and the state changes associated with eachtransaction.

The communications between the resource manager 102, client accounts,and principals to various transactions are often asynchronous. Theresource manager 102 generates a transaction identifier and a UUID(universal unique identifier) as a reference with each request to trackthe responses. The systems across the resource manager 102 areheterogenous and sometimes do not return the reference numbers back. Inthis case, the amounts, the positions, and the account number are usedto smartly match payments to the reference.

For payments that involve one-to-many or many-to-one payments, theprincipals may send the metadata about the net amounts. For example,clearinghouses may send all net metadata positions and CUSIPs. In manycases, there are likely several million CUSIPs held across multipleclearing members. The payment initiation from each clearing member'saccount is now associated with a different UUID. Each clearing membersends the trade positions of each of the clients that is associated with(or in communication with) the resource manager 102. The resourcemanager 102 initiates debits and credits from the accounts of theclients to and from the accounts of the clearing members. The resourcemanager 102 matches trade positions for the end clients and then ties inthe net payments of the trade position, fees, charges, and so forth.

FIG. 32 is a schematic flow chart diagram of a method 3200 fortransferring assets (e.g., funds) between two financial institutions.The method 3200 begins and the resource manager 102 receives at 3202 arequest to transfer funds from an account at Bank A to an account atBank B. The request may be received by Bank A, Bank B, or anotherfinancial institution, system, device, and the like. The resourcemanager 102 confirms at 3204 whether there are available funds for thetransfer. The resource manager 102 may confirm that an account at bank Acomprises sufficient funds to satisfy the amount of funds defined in thereceived transfer request. If available funds are confirmed at 3204, theresource manager 102 generates a suspense account A at Bank A andcreates a suspense account B at Bank B. The suspense account A andsuspense account B may be temporary suspense accounts created for aparticular transfer of funds. In other implementations, the suspenseaccount A and suspense account B are temporary suspense accounts but areused for a period of time (or for a number of transactions) to supporttransfers between bank A and bank B.

If available funds are confirmed at 2094, the account at Bank A isdebited at 3206 by the transfer amount and suspense account A (at BankA) is credited with the transfer amount. The resource manager 102 debitsthe transfer amount from the account at Bank A and credits that transferamount to the suspense account A. In some embodiments, ownership of thetransferred assets changes as soon as the transfer amount is credited tosuspense account A.

The transferred funds are settled at 3208 from suspense account A (atBank A) to suspense account B (at Bank B). The resource manager 102 maysettle funds from suspense account A in bank A to suspense account B inbank B. The settlement of funds between two suspense accounts isdetermined by the counterparty rules set up between the two financialinstitutions involved in the transfer of funds. For example, acounterparty may choose to settle at the top of the hour or at a certainthreshold to manage risk exposure. The settlement process may bedetermined by the asset type, the financial institution pair, and/or thetype of transaction. In some embodiments, transactions can be configuredto settle in gross or net. For gross transaction settlement of a PVPworkflow, the settlement occurs instantaneously over existing protocolssupported by financial institutions, such as FedWire, NSS, and the like.Netted transactions may also settle over existing protocols based oncounterparty and netting rules. In some embodiments, the funds aresettled after each funds transfer. In other embodiments, the funds aresettled periodically, such as once an hour or once a day. Thus, ratherthan settling the two suspense accounts after each funds transferbetween two financial institutions, the suspense accounts are settledafter multiple transfers that occur over a period of time.Alternatively, some embodiments settle the two suspense accounts whenthe amount due to one financial institution exceeds a threshold value.

The method 3200 continues as suspense account B (at Bank B) is debitedat 3210 by the transfer amount and account at Bank B is credited withthe transfer amount. After finishing step 3210, the funds transfer fromaccount A at bank A to account B at bank B is complete.

In some embodiments, the resource manager 102 facilitates (or initiates)the debit, credit, and settlement activities by sending appropriateinstructions to Bank A and/or Bank B. The appropriate bank then performsthe instructions to implement at least a portion of method 3200. Theexample of method 3200 can be performed with any type of asset. In someembodiments, the asset transfer is a transfer of funds using one or moretraditional currencies, such as U.S. Dollars (USD) or Great BritishPounds (GBP).

FIG. 33 illustrates an embodiment of a method 3300 for authenticating aclient and validating a transaction. Initially, the resource manager 102receives at 3302 a connection request from a client node, such as afinancial institution, an authorized system, an authorized user device,or other client types mentioned herein. The resource manager 102authenticates at 3304 and, if authenticated, acknowledges the clientnode as known. The method 3300 continues as the resource manager 102receives at 3306 a login request from the client node. In response tothe login request, the resource manager 102 generates at 3308 anauthentication token and communicates the authentication token to theclient node. In some embodiments, the authentication token is used todetermine the identity of the user for future requests, such as fundtransfer requests. The identity is then further checked for permissionsto the various services or actions.

The resource manager 102 further receives at 3310 a transaction requestfrom the client node, such as a request to transfer assets between twofinancial institutions or other entities. In response to the receivedtransaction request, the resource manager 102 verifies at 3312 theclient node's identity and validates the requested transaction. In someembodiments, the client node's identity is validated based on anauthentication token, and then permissions are checked to determine ifthe user has permissions to perform a particular action or transaction.Transfers of assets also involve validating approval of an account bymultiple roles to avoid compromising the network. If the client node'sidentity and requested transaction are verified, the resource managercreates 3314 one or more ledger entries to store the details of thetransaction. The ledger entries may be stored on the shared permissionedledger 110 as discussed herein. The resource manager 102 then sends at3316 an acknowledgement regarding the transaction to the client nodewith a server transaction token. In some embodiments, the servertransaction token is used at a future time by the client when conductingaudits. Finally, the resource manager 102 initiates at 3318 thetransaction using, for example, the systems and methods discussedherein.

As discussed herein, the described systems and methods facilitate themovement of assets between principals (also referred to as “parties” or“participants”). The principals are typically large financialinstitutions in capital markets that trade multiple financial products.Trades in capital markets can be complex and involve large assetmovements (also referred to as “settlements”). The systems and methodsdescribed herein can integrate to financial institutions and centralsettlement authorities such as the US Federal Reserve or DTCC(Depository Trust & Clearing Corporation) to facilitate the finalsettlement of assets. The described systems and methods also have theability to execute workflows such as DVP, threshold based settlement, ortime-based settlement between participants. Using the workflows,transactions are settled in gross or net amounts.

The systems and methods discussed herein include a hardware and/orsoftware platform that facilitates the movement of assets betweenprincipals. In some embodiments, the participants are large financialinstitutions in capital markets that trade multiple financial products.Trades in capital markets can be complex and involve large assetmovements (also referred to as “settlements”). The clearing andsettlement gateway discussed herein can integrate to financialinstitutions and central settlement authorities such as the U.S. FederalReserve, DTC, and the like to facilitate the final settlement of assets.

FIG. 34 is a schematic flow chart diagram of a method 3400 for nettingtransactions. In some embodiments, method 3400 is performed, at leastpartially, by the resource managers discussed herein. Initially, method3400 receives 3402 information associated with multiple trades. Thereceived information may be associated with any number of trades betweenany number of different parties using any number of differentcurrencies. Based on the received trade information, the methodcalculates 3404 the overall obligations and exposures by assets andcounterparties using predictive models. As discussed herein, thesepredictive models may include one or more stochastic trading liquiditymodels.

The method 3400 continues by identifying 3406 one or more thresholdsassociated with any number of transactions. In some embodiments, thesethresholds are based on particular parties and/or particular currencies.Example thresholds may define liquidity limits (e.g., minimumliquidity), exposure limits (e.g., maximum exposure), and the like. Forexample, a first party may set one or more threshold parametersassociated with a second party. Further, a first party may set one ormore threshold parameters associated with a particular currency. Thethresholds may be set by system administrators, parties, financialinstitutions, or other individuals or institutions.

The method 3400 further identifies 3408 existing trades across thetrading parties and the currencies associated with those trades. In aparticular example, multiple trades may be executed between multipleparties. A demand optimization engine selects at 3410 trades for aparticular netting cycle, which align with the overall bilateral nettedobligations and exposures between counterparties. For example, thedemand optimization engine analyzes configured obligations and exposurethresholds set in the system. Also, the demand optimization engine usespredictive models (e.g., stochastic trading liquidity models) tocalculate projected obligations and exposures. These inputs, along withthe set of existing trades across the trading parties and currencies areused to select the trades for the particular netting cycles. In someembodiments, the demand optimization engine considers or analyzes otherinputs and options for making trade selections, such as trade settlementpriority.

Using the selected trades, the demand optimization engine performs at3412 multiple bilateral netting operations (e.g., netting cycles) anduses the overall obligations and exposures as thresholds between tradingparties. In some embodiments, the demand optimization engine determinesthe most optimum route to settle the trades between partners. This maybe similar to route optimization and selects the order in which thenetting groups are settled between partners in any given cycle. In someembodiments, the demand optimization engine may settle multilaterally.In this situation, the multilateral nets are treated as FOP transactionswith all parties setting their transactions to complete the nettingcycle.

In many existing systems, transaction netting typically happens once perday, such as at the end of the business day. However, the systems andmethods discussed herein may be performed at any time of day and may beperformed multiple times during a particular day. For example, themethods discussed herein may be performed periodically each day tomonitor trades and predict future liquidity, exposure, and the like. Ifthe methods discussed herein identify a problem (e.g., based onthreshold values), action can be taken immediately instead of waitinguntil the end of the business day or some other future time. Thisperiodic monitoring can protect parties by identifying problems (orpotential problems) quickly instead of waiting for a one-a-day analysis.

FIG. 35 is a block diagram illustrating an example computing device3500. Computing device 3500 may be used to perform various procedures,such as those discussed herein. Computing device 3500 can function as aserver, a client, a client node, a resource manager, or any othercomputing entity. Computing device 3500 can be any of a wide variety ofcomputing devices, such as a workstation, a desktop computer, a notebookcomputer, a server computer, a handheld computer, a tablet, asmartphone, and the like. In some embodiments, computing device 3500represents any of the computing devices discussed herein.

Computing device 3500 includes one or more processor(s) 3502, one ormore memory device(s) 3504, one or more interface(s) 3506, one or moremass storage device(s) 3508, and one or more Input/Output (I/O)device(s) 3510, all of which are coupled to a bus 3512. Processor(s)3502 include one or more processors or controllers that executeinstructions stored in memory device(s) 3504 and/or mass storagedevice(s) 3508. Processor(s) 3502 may also include various types ofcomputer-readable media, such as cache memory.

Memory device(s) 3504 include various computer-readable media, such asvolatile memory (e.g., random access memory (RAM)) and/or nonvolatilememory (e.g., read-only memory (ROM)). Memory device(s) 3504 may alsoinclude rewritable ROM, such as Flash memory.

Mass storage device(s) 3508 include various computer readable media,such as magnetic tapes, magnetic disks, optical disks, solid statememory (e.g., Flash memory), and so forth. Various drives may also beincluded in mass storage device(s) 3508 to enable reading from and/orwriting to the various computer readable media. Mass storage device(s)3508 include removable media and/or non-removable media.

I/O device(s) 3510 include various devices that allow data and/or otherinformation to be input to or retrieved from computing device 3500.Example I/O device(s) 3510 include cursor control devices, keyboards,keypads, microphones, monitors or other display devices, speakers,printers, network interface cards, modems, lenses, CCDs or other imagecapture devices, and the like.

Interface(s) 3506 include various interfaces that allow computing device3500 to interact with other systems, devices, or computing environments.Example interface(s) 3506 include any number of different networkinterfaces, such as interfaces to local area networks (LANs), wide areanetworks (WANs), wireless networks, and the Internet.

Bus 3512 allows processor(s) 3502, memory device(s) 3504, interface(s)3506, mass storage device(s) 3508, and I/O device(s) 3510 to communicatewith one another, as well as other devices or components coupled to bus3512. Bus 3512 represents one or more of several types of busstructures, such as a system bus, PCI bus, IEEE 3594 bus, USB bus, andso forth.

For purposes of illustration, programs and other executable programcomponents are shown herein as discrete blocks, although it isunderstood that such programs and components may reside at various timesin different storage components of computing device 3500 and areexecuted by processor(s) 3502. Alternatively, the systems and proceduresdescribed herein can be implemented in hardware, or a combination ofhardware, software, and/or firmware. For example, one or moreapplication specific integrated circuits (ASICs) can be programmed tocarry out one or more of the systems and procedures described herein.

Examples

The following examples pertain to further embodiments.

Example 1 is a system. The system includes a resource manager incommunication with a network, wherein the resource manager comprises adata ingestion engine and a netting module. The system includes anexecution platform comprising a plurality of processing nodes, whereineach of the plurality of processing nodes is assigned to one clientaccount of a plurality of client accounts coupled to the resourcemanager. The system includes a shared permissioned ledger comprising aplurality of ledger instances, wherein each of the plurality of ledgerinstances is assigned to one client account of the plurality of clientaccounts, and wherein storage resources on the shared permissionedledger are independently scalable from processing resources on theexecution platform. The system is such that the data ingestion enginecomprises a plurality of node-specific ingestors, wherein each of theplurality of node-specific ingestors is assigned to a data stream eventchannel pushed by one of the plurality of client accounts. The system issuch that each of the plurality of node-specific ingestors feeds datafrom the assigned data stream even channel to an assigned node-specificnormalizer configured to normalize the data. The system is such that thenetting module is executed on independent processing nodes for each ofthe plurality of client accounts to identify one or more trades to beincluded in a netting group based on the normalized data.

Example 2 is a system as in Example 1, wherein the resource manager iscoupled to a first client account and a second client account, andwherein: the execution platform comprises a first processing nodeassigned to the first client account and a second processing nodeassigned to the second client account; the shared permissioned ledgercomprises a first ledger instance assigned to the first client accountand a second ledger instance assigned to the second client account; thefirst processing node executes a first instance of the data ingestionengine for consuming data pushed by the first client account, whereinthe first instance of the data ingestion engine comprises one or morenode-specific ingestors and one or more node-specific normalizers eachassigned to one data stream event channel pushed by the first clientaccount; the second processing node executes a second instance of thedata ingestion engine for consuming data pushed by the second clientaccount, wherein the second instance of the data ingestion enginecomprises one or more node-specific ingestors and one or morenode-specific normalizers each assigned to one data stream event channelpushed by the second client account.

Example 3 is a system as in any of Examples 1-2, wherein the resourcemanager scales the processing resources on the execution platform andthe storage resources on the shared permissioned ledger up and down tothe first processing node, the second processing node, the first ledgerinstance, and the second ledger instance based on client need.

Example 4 is a system as in any of Examples 1-3, wherein the sharedpermissioned ledger stores normalized data entries comprising trade dataassociated with the plurality of client accounts, and wherein: the firstledger instance stores trade data only associated with the first clientaccount; the second ledger instance stores trade data only associatedwith the second client account; the first processing node does not haveread or write authorization on the second ledger instance; and thesecond processing node does not have read or write authorization on thefirst ledger instance.

Example 5 is a system as in any of Examples 1-4, wherein the one or morenode-specific normalizers of the first processing node push normalizeddata to a first normalized data channel, and wherein the one or morenode-specific normalizers of the second processing node push normalizeddata to a second normalized data channel, and wherein: the firstprocessing node executes a first instance of the netting module, whereinthe first instance of the netting module reads the first normalized datachannel and does not have authorization to the second normalized datachannel; and the second processing node executes a second instance ofthe netting module, wherein the second instance of the netting modulereads the second normalized data channel and does not have authorizationto read the first normalized data channel.

Example 6 is a system as in any of Examples 1-5, wherein each of thefirst instance of the netting module and the second instance of thenetting module execute netting instructions for calculating nettingobligations, wherein the first instance of the netting module calculatesnetting obligations for the first client account, and wherein the secondinstance of the netting module calculates netting obligations for thesecond client account, and wherein the netting instructions comprise:determining a most recent netting cycle based on data stored on theshared permissioned ledger, wherein the most recent netting cyclecomprises trades wherein the first client account and the second clientaccount are counterparties; identifying one or more pending tradesbetween the first client account and the second client account since themost recent netting cycle; generating a current netting group comprisingthe one or more pending trades since the most recent netting cycle; anddynamically updating the current netting group with new trades betweenthe first client account and the second client account based on datareceived from the first normalized data channel and/or the secondnormalized data channel.

Example 7 is a system as in any of Examples 1-6, wherein the nettinginstructions further comprise determining when the current netting groupshould be closed and settled based on rules-based triggers andspecifications set by the first client account and/or the second clientaccount comprising one or more of: a predetermined time and/or date forsettling netting groups, a trade-quantity risk profile, a trade-valuerisk profile, a liquidity threshold, or an output of a stochasticpredictive model for calculating future obligations and exposures.

Example 8 is a system as in any of Examples 1-7, wherein the nettinginstructions further comprise: assigning a netting ID to the currentnetting group; identifying a trade ID for trades within the currentnetting group; causing updated data entries to be stored on the sharedpermissioned ledger for each trade within the netting group, wherein theupdated data entries comprise the netting ID and an applicable trade ID.

Example 9 is a system as in any of Examples 1-8, wherein the nettinginstructions further comprise executing the stochastic predictive modelto predict future obligations and exposures based on historical data,wherein executing the stochastic predictive model comprises calculatinga predictive quantity for each asset type traded within the currentnetting group at a future time.

Example 10 is a system as in any of Examples 1-9, wherein each of theplurality of client account represents a financial institutioncomprising one or more of a bank, credit union, hedge fund, assetmanagement system, asset management organization, mutual fund,clearinghouse, or exchange, and wherein the financial institution pushesfinancial trade data to the data ingestion engine.

Example 11 is a system as in any of Examples 1-10, wherein the dataingestion engine receives financial trade in a plurality of dataformats, and wherein the plurality of node-specific normalizers comprisea software module for translating ingested raw data from a languagedefined by the applicable client account to a canonical format used bythe resource manager.

Example 12 is a system as in any of Examples 1-11, wherein the nettingmodule calculates bilateral netting for two parties and furthercalculates multilateral netting for three or more parties in asettlement group.

Example 13 is a system as in any of Examples 1-12, wherein the pluralityof processing nodes are configured to calculate trade splits for anassigned client account, wherein the trade split comprises an indicationof how many trade-let executions should be executed to settle a trade infull.

Example 14 is a system as in any of Examples 1-13, wherein calculatingthe trade splits comprises suggesting a trade split based on one or moreof: obligations and exposures of the assigned client account for anasset type applicable to a certain trade; current liquidity of theassigned client account; predicted liquidity of a counterparty to thecertain trade based on an output from a stochastic liquidity model;permissible lot size as defined by the assigned client account; one ormore risk thresholds or liquidity thresholds defined by the assignedclient account; a number of settlement cycles remaining in a definedtime period; a number of pending trades associated with the assignedclient account; or a number of pending trade orders left to settled in adefined time period.

Example 15 is a system as in any of Examples 1-14, wherein theprocessing node is further configured to provide the suggested tradesplit to one or more counterparties for the certain trade for approvalor denial by the one or more counterparties.

Example 16 is a system as in any of Examples 1-15, wherein the resourcemanager further comprises a liquidity router for calculating alowest-cost pathway for executing a currency exchange, wherein thelowest-cost pathway comprises one or more of: a lowest cost based oncurrency exchange rate losses or a lowest-cost based on fewest number ofhop trades.

Example 17 is a system as in any of Examples 1-16, wherein each of theplurality of client accounts engaging in currency exchange comprises anindependent liquidity router, wherein each of the independent liquidityrouters is assigned to one client account of the plurality of clientaccounts such that the independent liquidity routers can only accessdata stored on the ledger instance assigned to the one client to whichthe independent liquidity router is assigned.

Example 18 is a system as in any of Examples 1-17, wherein the liquidityrouter executes a currency predictive model for calculating thelowest-cost pathway for executing the currency exchange, wherein thecurrency predictive model is a stochastic model for predicting currentand future liquidity of a plurality of currencies based on one or moreof: current currency positions of counterparties to a trade, currentliquidity for a plurality of currencies, least cross, historical bestrates for the plurality of currencies, and an identification of marketmakers likely to have liquidity in any of the plurality of currencies,and wherein the currency predictive model outputs results to a pseudoledger.

Example 19 is a system as in any of Examples 1-18, wherein the liquidityrouter calculates the lowest-cost pathway for executing the currencyexchange based on outputs stored on the pseudo ledger and by executing ashortest path algorithm in graph theory, wherein: a first algorithm nodeindicates an initial currency in the currency exchange; a secondalgorithm node indicates a final currency in the currency exchange; oneor more intermediary nodes indicate currency pairs that can beexchanged; and edges between nodes in the shortest path algorithmindicate a back-to-back trade between two ledgers for an applicablecurrency pair.

Example 20 is a system as in any of Examples 1-19, wherein one or moreof the initial currency or the final currency in the currency exchangeis an exotic currency, wherein exotic currencies comprises non-G10currencies.

Example 21 is a system. The system includes a resource manager incommunication with a network. The system includes an execution platformcomprising a plurality of processing nodes, wherein each of theplurality of processing nodes is assigned to one client account of aplurality of client accounts coupled to the resource manager. The systemincludes a shared permissioned ledger comprising a plurality of ledgerinstances, wherein each of the plurality of ledger instances is assignedto one client account of the plurality of client accounts. The system issuch that the resource manager comprises one or more processors forexecuting instructions stored in non-transitory computer readablestorage media. The instructions include scaling processing resources onthe execution platform independently from scaling storage resources onthe shared permissioned ledger based on need. The instructions includedefining a settlement group comprising one or more settlements clientsfrom the plurality of client accounts, wherein the settlement groupcomprises trading rules that apply to each of the one or more settlementclients. The instructions include ingesting data from each of theplurality of client accounts by way of a data ingestion enginecomprising a plurality of node-specific ingestors, wherein each of theplurality of node-specific ingestors is assigned to a data stream eventchannel pushed by one of the plurality of client accounts. Theinstructions include assigning the ingested data an encryption level ona key hierarchy within the shared permissioned ledger based on contentof the ingested data. The instructions include authenticating anobserver node associated with at least one of the one or more settlementclients, wherein the observer node is granted read access to certaindata on the shared permissioned ledger.

Example 22 is a system as in Example 21, wherein the instructionsfurther comprise: ingesting data from each of the plurality of clientaccounts by way of the data ingestion engine; normalizing the ingesteddata to a canonical format readable by the resource manager to generatenormalized data; partitioning the normalized data across the pluralityof ledger instances of the shared permissioned ledger based onownership, wherein a first client account assigned to a first ledgerinstance can only read or write to data stored on the first ledgerinstance.

Example 23 is a system as in any of Examples 21-22, wherein theinstructions further comprise: assigning the normalized data anencryption level on the key hierarchy of the shared permissioned levelbased on content of the normalized data; encrypting the normalized datawith an appropriate encryption key based on the assigned encryptionlevel on the key hierarchy; and assigning permissions to the normalizeddata based on the assigned encryption level on the key hierarchy.

Example 24 is a system as in any of Examples 21-23, wherein theinstructions are such that encrypting the normalized data comprises:identifying an owner of a normalized data entry based on whichnode-specific ingestor consumed the normalized data entry; determiningwhether the normalized data entry will be shared with a node that doesnot have ownership of the normalized data entry; receiving an encryptionkey from a third party service, wherein the encryption key is assignedto the owner of the normalized data entry; and in response todetermining the normalized data entry will be shared with a non-owner,encrypting the normalized data entry with the encryption key.

Example 25 is a system as in any of Examples 21-24, wherein the keyhierarchy for the shared permissioned ledger comprises: level zero keysapplied to data comprising one or more of a group identifier, a tradedata fee, trade data, or a trade identifier; level one keys applied todata comprising a trade set identifier; level two keys assigned to datacomprising one or more of a trade bundle keyset or a trade bundleidentifier; and level three keys assigned to data comprising one or moreof a trade bag keyset or a trade bag identifier.

Example 26 is a system as in any of Examples 21-25, wherein the observernode is associated with one or more of a clearinghouse, exchange,settlement agent, or regulatory agency, and wherein the instructionsfurther comprise: generating a data subscription model for the observernode based on permissions granted to the observer node by an associatedsettlement client, wherein the data subscription model identifies one ormore events occurring within the settlement group; triggering the datasubscription model when any of the one or more events occurs within thesettlement group; identifying data stored on a ledger instanceassociated with the settlement client that is part of the datasubscription model; and providing the identified data to the observernode by way of a data push notification and/or a data pullauthorization.

Example 27 is a system as in any of Examples 21-26, wherein theinstructions further comprise generating a reconciliation report for theassociated settlement client, wherein generating the reconciliationreport comprises: identifying trade data to be included in thereconciliation report; hashing the trade data with a unique hash basedon a public key assigned to the associated settlement client; storingthe unique hash on a datastore independent of the shared permissionedledger; and digitally signing the reconciliation report with a privatekey, wherein the private key ensures the reconciliation report isavailable only to the associated settlement client and the observernode.

Example 28 is a system as in any of Examples 21-27, wherein theinstructions further comprise: authorizing a suspense account andassociating the suspense account with the settlement group, whereinassets traded between the one or more settlement clients of thesettlement group are traded within the suspense account; and trackingownership of assets within the suspense account with data entries storedacross the plurality of ledger instances of the shared permissionedledger.

Example 29 is a system as in any of Examples 21-28, wherein theinstructions further comprise: receiving a request to execute atransaction between a first settlement client and a second settlementclient; determining whether sufficient assets exist in the suspenseaccount belonging to the first settlement client and/or the secondsettlement client for executing the transaction; in response todetermining that sufficient assets exist to execute the transaction,generating one or more data entries on the shared permissioned ledgerindicating the assets have transferred ownership within the suspenseaccount according to the requested transaction.

Example 30 is a system as in any of Examples 21-29, wherein the firstsettlement client is associated with a first ledger instance of theshared permissioned ledger, and wherein the second settlement client isassociated with a second ledger instance of the shared permissionedledger, and wherein the instructions are such that generating the one ormore data entries comprises: generating a first data entry to be storedon the first ledger instance, wherein the first data entry representsthe transaction from a point of view of the first settlement client; andgenerating a second data entry to be stored on the second ledgerinstance, wherein the second data entry represents the transaction froma point of view of the second settlement client; wherein the firstsettlement client does not have read or write authorization on thesecond ledger instance; and wherein the second settlement client doesnot have read or write authorization on the first ledger instance.

Example 31 is a system as in any of Examples 21-30, wherein theinstructions are such that the transaction can be executed at any timebased on data entries stored on the shared permissioned ledger such thatthe transaction can be settled during non-operating hours for a bankassociated with the suspense account.

Example 32 is a system as in any of Examples 21-31, wherein the resourcemanager is coupled to a first client account and a second clientaccount, and wherein: the execution platform comprises a firstprocessing node assigned to the first client account and a secondprocessing node assigned to the second client account; the sharedpermissioned ledger comprises a first ledger instance assigned to thefirst client account and a second ledger instance assigned to the secondclient account; the first processing node executes a first instance ofthe data ingestion engine for consuming data pushed by the first clientaccount, wherein the first instance of the data ingestion enginecomprises one or more node-specific ingestors and one or morenode-specific normalizers each assigned to one data stream event channelpushed by the first client account; and the second processing nodeexecutes a second instance of the data ingestion engine for consumingdata pushed by the second client account, wherein the second instance ofthe data ingestion engine comprises one or more node-specific ingestorsand one or more node-specific normalizers each assigned to one datastream event channel pushed by the second client account.

Example 33 is a system as in any of Examples 21-32, wherein the sharedpermissioned ledger stores normalized data entries comprising trade dataassociated with the plurality of client accounts, and wherein: the firstledger instance stores trade data only associated with the first clientaccount; the second ledger instance stores trade data only associatedwith the second client account; the first processing node does not haveread or write authorization on the second ledger instance; and thesecond processing node does not have read or write authorization on thefirst ledger instance.

Example 34 is a system as in any of Examples 21-33, further comprising anormalization module for normalizing data consumed by the data ingestionengine, and wherein one or more node-specific normalizers of the firstprocessing node push normalized data to a first normalized data channel,and wherein one or more node-specific normalizers of the secondprocessing node push normalized data to a second normalized datachannel, and wherein: the first processing node executes a firstinstance of a reconciliation module, wherein the first instance of thenetting module reads the first normalized data channel and does not haveauthorization to the second normalized data channel; and the secondprocessing node executes a second instance of the reconciliation module,wherein the second instance of the netting module reads the secondnormalized data channel and does not have authorization to read thefirst normalized data channel.

Example 35 is a system as in any of Examples 21-34, wherein each of theplurality of client account represents a financial institutioncomprising one or more of a bank, credit union, hedge fund, assetmanagement system, asset management organization, mutual fund,clearinghouse, or exchange, and wherein the financial institution pushesfinancial trade data to the data ingestion engine.

Example 36 is a system as in any of Examples 21-35, wherein the dataingestion engine receives financial trade data in a plurality of dataformats, and wherein the plurality of node-specific ingestors comprise asoftware module for translating ingested raw data from a languagedefined by the applicable client account to a canonical format used bythe resource manager.

Example 37 is a system as in any of Examples 21-36, wherein theinstructions further comprise: identifying a first settlement clientwithin the settlement group; identifying clearinghouse-level data ownedby the first settlement client and stored on a ledger instanceassociated with the first settlement client, wherein theclearinghouse-level data is assigned permissions to be shared with anauthorized observer node; identifying bankruptcy-specific data stored onthe ledger instance associated with the first settlement client, whereinthe bankruptcy-specific data comprises obligations of the firstsettlement client to its counterparties; and identifying confidentialdata stored on the ledger instance associated with the first settlementclient, wherein the confidential data comprises permissions andencryptions indicating the confidential data will not be shared withother parties.

Example 38 is a system as in any of Examples 21-37, wherein theinstructions further comprise: providing the clearinghouse-level data toan authorized observer node with permission to view theclearinghouse-level data; identifying a second settlement client withinthe settlement group with authorization to view the bankruptcy-specificdata associated with the first settlement client; enabling the secondsettlement client to read the bankruptcy-specific data associated withthe first settlement client; and ensuring the confidential data is notreplicated.

Example 39 is a system as in any of Examples 21-38, wherein theinstructions further comprise: generating a first reconciliation reportfor the first settlement client based on the clearinghouse-level storedon the ledger instance associated with the first settlement client,wherein the first reconciliation report reconciles net debits andcredits for the one or more settlement clients within the settlementgroup; and generating a second reconciliation report for the secondsettlement client based on clearinghouse-level stored on a ledgerinstance associated with the second settlement client, wherein thesecond reconciliation report reconciles net debits and credits for theone or more settlement clients within the settlement group; wherein thefirst reconciliation report and the second reconciliation report aregenerated according to the key hierarchy.

Example 40 is a system as in any of Examples 21-39, wherein theinstructions further comprise: determining whether a conflict existsbetween the first reconciliation report and the second reconciliationreport by determining whether a first hash associated with the firstsettlement client matches a second hash associated with the settlementclient; in response to determining the first hash does not match thesecond hash, identifying a common set of documents between the firstsettlement client and the second settlement client; and identifyingdifferences between common documents by hashing data rows from thecommon documents into tables and identifying rows that do not match.

In the above disclosure, reference has been made to the accompanyingdrawings, which form a part hereof, and in which is shown by way ofillustration specific implementations in which the disclosure may bepracticed. It is understood that other implementations may be utilized,and structural changes may be made without departing from the scope ofthe present disclosure. References in the specification to “oneembodiment,” “an embodiment,” “an example embodiment,” “selectedembodiments,” “certain embodiments,” etc., indicate that the embodimentor embodiments described may include a particular feature, structure, orcharacteristic, but every embodiment may not necessarily include theparticular feature, structure, or characteristic. Additionally, suchphrases are not necessarily referring to the same embodiment. Further,when a particular feature, structure, or characteristic is described inconnection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to affect such feature, structure,or characteristic in connection with other embodiments whether or notexplicitly described.

Implementations of the systems, devices, and methods disclosed hereinmay comprise or utilize a special purpose or general-purpose computerincluding computer hardware, such as, for example, one or moreprocessors and system memory, as discussed herein. Implementationswithin the scope of the present disclosure may also include physical andother computer-readable media for carrying or storingcomputer-executable instructions and/or data structures. Suchcomputer-readable media can be any available media that may be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arecomputer storage media (devices). Computer-readable media that carrycomputer-executable instructions are transmission media. Thus, by way ofexample, and not limitation, implementations of the disclosure caninclude at least two distinctly different kinds of computer-readablemedia: computer storage media (devices) and transmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM,solid state drives (“SSDs”) (e.g., based on RANI), Flash memory,phase-change memory (“PCM”), other types of memory, other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store desired program code means inthe form of computer-executable instructions or data structures andwhich can be accessed by a general purpose or special purpose computer.

An implementation of the devices, systems, and methods disclosed hereinmay communicate over a computer network. A “network” is defined as oneor more data links that enable the transport of electronic data betweencomputer systems and/or modules and/or other electronic devices. Wheninformation is transferred or provided over a network or anothercommunications connection (either hardwired, wireless, or a combinationof hardwired and wireless) to a computer, the computer properly viewsthe connection as a transmission medium. Transmissions media can includea network and/or data links, which can be used to carry desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer. Combinations of the above should also be includedwithin the scope of computer-readable media.

Computer-executable instructions include, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. Thecomputer-executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, various storage devices, andthe like. The disclosure may also be practiced in distributed systemenvironments where local and remote computer systems, which are linked(either by hardwired data links, wireless data links, or by acombination of hardwired and wireless data links) through a network,both perform tasks. In a distributed system environment, program modulesmay be located in both local and remote memory storage devices.

Further, where appropriate, functions described herein can be performedin one or more of: hardware, software, firmware, digital components, oranalog components. For example, one or more application specificintegrated circuits (ASICs) can be programmed to carry out one or moreof the systems and procedures described herein. Certain terms are usedthroughout the description and claims to refer to particular systemcomponents. As one skilled in the art will appreciate, components may bereferred to by different names. This document does not intend todistinguish between components that differ in name, but not function.

It should be noted that the sensor embodiments discussed above maycomprise computer hardware, software, firmware, or any combinationthereof to perform at least a portion of their functions. For example, amodule may include computer code configured to be executed in one ormore processors and may include hardware logic/electrical circuitrycontrolled by the computer code. These example devices are providedherein purposes of illustration and are not intended to be limiting.Embodiments of the present disclosure may be implemented in furthertypes of devices, as would be known to persons skilled in the relevantart(s).

At least some embodiments of the disclosure have been directed tocomputer program products comprising such logic (e.g., in the form ofsoftware) stored on any computer useable medium. Such software, whenexecuted in one or more data processing devices, causes a device tooperate as described herein.

While various embodiments of the present disclosure are describedherein, it should be understood that they are presented by way ofexample only, and not limitation. It will be apparent to persons skilledin the relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the disclosure.Thus, the breadth and scope of the present disclosure should not belimited by any of the described exemplary embodiments but should bedefined only in accordance with the following claims and theirequivalents. The description herein is presented for the purposes ofillustration and description. It is not intended to be exhaustive or tolimit the disclosure to the precise form disclosed. Many modificationsand variations are possible in light of the disclosed teaching. Further,it should be noted that any or all of the alternate implementationsdiscussed herein may be used in any combination desired to formadditional hybrid implementations of the disclosure.

What is claimed is:
 1. A system comprising: a resource manager incommunication with a network; an execution platform comprising aplurality of processing nodes, wherein each of the plurality ofprocessing nodes is assigned to one client account of a plurality ofclient accounts coupled to the resource manager; and a sharedpermissioned ledger comprising a plurality of ledger instances, whereineach of the plurality of ledger instances is assigned to one clientaccount of the plurality of client accounts; wherein the resourcemanager comprises one or more processors for executing instructionsstored in non-transitory computer readable storage media, theinstructions comprising: scaling processing resources on the executionplatform independently from scaling storage resources on the sharedpermissioned ledger based on need; defining a settlement groupcomprising one or more settlements clients from the plurality of clientaccounts, wherein the settlement group comprises trading rules thatapply to each of the one or more settlement clients; ingesting data fromeach of the plurality of client accounts by way of a data ingestionengine comprising a plurality of node-specific ingestors, wherein eachof the plurality of node-specific ingestors is assigned to a data streamevent channel pushed by one of the plurality of client accounts;assigning the ingested data an encryption level on a key hierarchywithin the shared permissioned ledger based on content of the ingesteddata; and authenticating an observer node associated with at least oneof the one or more settlement clients, wherein the observer node isgranted read access to certain data on the shared permissioned ledger.2. The system of claim 1, wherein the instructions further comprise:ingesting data from each of the plurality of client accounts by way ofthe data ingestion engine; normalizing the ingested data to a canonicalformat readable by the resource manager to generate normalized data;partitioning the normalized data across the plurality of ledgerinstances of the shared permissioned ledger based on ownership, whereina first client account assigned to a first ledger instance can only reador write to data stored on the first ledger instance.
 3. The system ofclaim 2, wherein the instructions further comprise: assigning thenormalized data an encryption level on the key hierarchy of the sharedpermissioned level based on content of the normalized data; encryptingthe normalized data with an appropriate encryption key based on theassigned encryption level on the key hierarchy; and assigningpermissions to the normalized data based on the assigned encryptionlevel on the key hierarchy.
 4. The system of claim 3, wherein theinstructions are such that encrypting the normalized data comprises:identifying an owner of a normalized data entry based on whichnode-specific ingestor consumed the normalized data entry; determiningwhether the normalized data entry will be shared with a node that doesnot have ownership of the normalized data entry; receiving an encryptionkey from a third party service, wherein the encryption key is assignedto the owner of the normalized data entry; and in response todetermining the normalized data entry will be shared with a non-owner,encrypting the normalized data entry with the encryption key.
 5. Thesystem of claim 1, wherein the key hierarchy for the shared permissionedledger comprises: level zero keys applied to data comprising one or moreof a group identifier, a trade data fee, trade data, or a transactionidentifier; level one keys applied to data comprising a trade setidentifier; level two keys assigned to data comprising one or more of atrade bundle keyset or a trade bundle identifier; and level three keysassigned to data comprising one or more of a trade bag keyset or a tradebag identifier.
 6. The system of claim 1, wherein the observer node isassociated with one or more of a clearinghouse, exchange, settlementagent, or regulatory agency, and wherein the instructions furthercomprise: generating a data subscription model for the observer nodebased on permissions granted to the observer node by an associatedsettlement client, wherein the data subscription model identifies one ormore events occurring within the settlement group; triggering the datasubscription model when any of the one or more events occurs within thesettlement group; identifying data stored on a ledger instanceassociated with the settlement client that is part of the datasubscription model; and providing the identified data to the observernode by way of a data push notification and/or a data pullauthorization.
 7. The system of claim 6, wherein the instructionsfurther comprise generating a reconciliation report for the associatedsettlement client, wherein generating the reconciliation reportcomprises: identifying trade data to be included in the reconciliationreport; hashing the trade data with a unique hash based on a public keyassigned to the associated settlement client; storing the unique hash ona datastore independent of the shared permissioned ledger; and digitallysigning the reconciliation report with a private key, wherein theprivate key ensures the reconciliation report is available only to theassociated settlement client and the observer node.
 8. The system ofclaim 1, wherein the instructions further comprise: authorizing asuspense account and associating the suspense account with thesettlement group, wherein assets traded between the one or moresettlement clients of the settlement group are traded within thesuspense account; and tracking ownership of assets within the suspenseaccount with data entries stored across the plurality of ledgerinstances of the shared permissioned ledger.
 9. The system of claim 8,wherein the instructions further comprise: receiving a request toexecute a transaction between a first settlement client and a secondsettlement client; determining whether sufficient assets exist in thesuspense account belonging to the first settlement client and/or thesecond settlement client for executing the transaction; in response todetermining that sufficient assets exist to execute the transaction,generating one or more data entries on the shared permissioned ledgerindicating the assets have transferred ownership within the suspenseaccount according to the requested transaction.
 10. The system of claim9, wherein the first settlement client is associated with a first ledgerinstance of the shared permissioned ledger, and wherein the secondsettlement client is associated with a second ledger instance of theshared permissioned ledger, and wherein the instructions are such thatgenerating the one or more data entries comprises: generating a firstdata entry to be stored on the first ledger instance, wherein the firstdata entry represents the transaction from a point of view of the firstsettlement client; and generating a second data entry to be stored onthe second ledger instance, wherein the second data entry represents thetransaction from a point of view of the second settlement client;wherein the first settlement client does not have read or writeauthorization on the second ledger instance; and wherein the secondsettlement client does not have read or write authorization on the firstledger instance.
 11. The system of claim 9, wherein the instructions aresuch that the transaction can be executed at any time based on dataentries stored on the shared permissioned ledger such that thetransaction can be settled during non-operating hours for a bankassociated with the suspense account.
 12. The system of claim 1, whereinthe resource manager is coupled to a first client account and a secondclient account, and wherein: the execution platform comprises a firstprocessing node assigned to the first client account and a secondprocessing node assigned to the second client account; the sharedpermissioned ledger comprises a first ledger instance assigned to thefirst client account and a second ledger instance assigned to the secondclient account; the first processing node executes a first instance ofthe data ingestion engine for consuming data pushed by the first clientaccount, wherein the first instance of the data ingestion enginecomprises one or more node-specific ingestors and one or morenode-specific normalizers each assigned to one data stream event channelpushed by the first client account; and the second processing nodeexecutes a second instance of the data ingestion engine for consumingdata pushed by the second client account, wherein the second instance ofthe data ingestion engine comprises one or more node-specific ingestorsand one or more node-specific normalizers each assigned to one datastream event channel pushed by the second client account.
 13. The systemof claim 12, wherein the shared permissioned ledger stores normalizeddata entries comprising trade data associated with the plurality ofclient accounts, and wherein: the first ledger instance stores tradedata only associated with the first client account; the second ledgerinstance stores trade data only associated with the second clientaccount; the first processing node does not have read or writeauthorization on the second ledger instance; and the second processingnode does not have read or write authorization on the first ledgerinstance.
 14. The system of claim 13, further comprising a normalizationmodule for normalizing data consumed by the data ingestion engine, andwherein one or more node-specific normalizers of the first processingnode push normalized data to a first normalized data channel, andwherein one or more node-specific normalizers of the second processingnode push normalized data to a second normalized data channel, andwherein: the first processing node executes a first instance of areconciliation module, wherein the first instance of the netting modulereads the first normalized data channel and does not have authorizationto the second normalized data channel; and the second processing nodeexecutes a second instance of the reconciliation module, wherein thesecond instance of the netting module reads the second normalized datachannel and does not have authorization to read the first normalizeddata channel.
 15. The system of claim 1, wherein each of the pluralityof client account represents a financial institution comprising one ormore of a bank, credit union, hedge fund, asset management system, assetmanagement organization, mutual fund, clearinghouse, or exchange, andwherein the financial institution pushes financial trade data to thedata ingestion engine.
 16. The system of claim 1, wherein the dataingestion engine receives financial trade data in a plurality of dataformats, and wherein the plurality of node-specific ingestors comprise asoftware module for translating ingested raw data from a languagedefined by the applicable client account to a canonical format used bythe resource manager.
 17. The system of claim 1, wherein theinstructions further comprise: identifying a first settlement clientwithin the settlement group; identifying clearinghouse-level data ownedby the first settlement client and stored on a ledger instanceassociated with the first settlement client, wherein theclearinghouse-level data is assigned permissions to be shared with anauthorized observer node; identifying bankruptcy-specific data stored onthe ledger instance associated with the first settlement client, whereinthe bankruptcy-specific data comprises obligations of the firstsettlement client to its counterparties; and identifying confidentialdata stored on the ledger instance associated with the first settlementclient, wherein the confidential data comprises permissions andencryptions indicating the confidential data will not be shared withother parties.
 18. The system of claim 17, wherein the instructionsfurther comprise: providing the clearinghouse-level data to anauthorized observer node with permission to view the clearinghouse-leveldata; identifying a second settlement client within the settlement groupwith authorization to view the bankruptcy-specific data associated withthe first settlement client; enabling the second settlement client toread the bankruptcy-specific data associated with the first settlementclient; and ensuring the confidential data is not replicated.
 19. Thesystem of claim 18, wherein the instructions further comprise:generating a first reconciliation report for the first settlement clientbased on the clearinghouse-level stored on the ledger instanceassociated with the first settlement client, wherein the firstreconciliation report reconciles net debits and credits for the one ormore settlement clients within the settlement group; and generating asecond reconciliation report for the second settlement client based onclearinghouse-level stored on a ledger instance associated with thesecond settlement client, wherein the second reconciliation reportreconciles net debits and credits for the one or more settlement clientswithin the settlement group; wherein the first reconciliation report andthe second reconciliation report are generated according to the keyhierarchy.
 20. The system of claim 19, wherein the instructions furthercomprise: determining whether a conflict exists between the firstreconciliation report and the second reconciliation report bydetermining whether a first hash associated with the first settlementclient matches a second hash associated with the settlement client; inresponse to determining the first hash does not match the second hash,identifying a common set of documents between the first settlementclient and the second settlement client; and identifying differencesbetween common documents by hashing data rows from the common documentsinto tables and identifying rows that do not match.